Note: This is test shard 3 of 6.
[==========] Running 7 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from RaftConsensusITest
[ RUN ] RaftConsensusITest.TestFailedOp
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260809 19:20:46.420379 4039 test_util.cc:268] Using random seed: 578210114
I20260809 19:20:46.426985 4039 ts_itest-base.cc:112] Starting cluster with:
I20260809 19:20:46.427042 4039 ts_itest-base.cc:113] --------------
I20260809 19:20:46.427054 4039 ts_itest-base.cc:114] 3 tablet servers
I20260809 19:20:46.427071 4039 ts_itest-base.cc:115] 3 replicas per TS
I20260809 19:20:46.427081 4039 ts_itest-base.cc:116] --------------
2026-08-09T19:20:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:20:46Z Disabled control of system clock
I20260809 19:20:46.438423 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.241.254:33225
--webserver_interface=127.3.241.254
--webserver_port=0
--builtin_ntp_servers=127.3.241.212:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.241.254:33225 with env {}
W20260809 19:20:46.526204 4058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:46.526418 4058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:46.526443 4058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:46.527655 4058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:20:46.527700 4058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:46.527715 4058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:20:46.527740 4058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:20:46.529175 4058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.241.254:33225
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.241.254:33225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.3.241.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.4058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:46.529385 4058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:46.529692 4058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:46.535058 4069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:46.535619 4076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:46.536084 4067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:46.536466 4058 server_base.cc:1034] running on GCE node
I20260809 19:20:46.536736 4058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:46.537050 4058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:46.538195 4058 hybrid_clock.cc:648] HybridClock initialized: now 1786303246538172 us; error 83 us; skew 500 ppm
I20260809 19:20:46.539563 4058 webserver.cc:460] Webserver started at http://127.3.241.254:38095/ using document root <none> and password file <none>
I20260809 19:20:46.539819 4058 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:46.539901 4058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:46.540052 4058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:46.541327 4058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "c69b1f11c3f1451585873b6f5e1b76ee"
format_stamp: "Formatted at 2026-08-09 19:20:46 on dist-test-slave-gmjp"
I20260809 19:20:46.541954 4058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "c69b1f11c3f1451585873b6f5e1b76ee"
format_stamp: "Formatted at 2026-08-09 19:20:46 on dist-test-slave-gmjp"
I20260809 19:20:46.543620 4058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:20:46.544503 4083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:46.544723 4058 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:46.544797 4058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/wal
uuid: "c69b1f11c3f1451585873b6f5e1b76ee"
format_stamp: "Formatted at 2026-08-09 19:20:46 on dist-test-slave-gmjp"
I20260809 19:20:46.544932 4058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:46.557010 4058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:46.557358 4058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:46.557509 4058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:46.562397 4058 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.254:33225
I20260809 19:20:46.562430 4166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.254:33225 every 8 connection(s)
I20260809 19:20:46.562764 4058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260809 19:20:46.563407 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 4058
I20260809 19:20:46.563496 4039 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260809 19:20:46.563733 4167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:46.567379 4167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee: Bootstrap starting.
I20260809 19:20:46.568260 4167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:46.568627 4167 log.cc:824] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:46.569562 4167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee: No bootstrap required, opened a new log
I20260809 19:20:46.571815 4167 raft_consensus.cc:348] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69b1f11c3f1451585873b6f5e1b76ee" member_type: VOTER last_known_addr { host: "127.3.241.254" port: 33225 } }
I20260809 19:20:46.572067 4167 raft_consensus.cc:374] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:46.572104 4167 raft_consensus.cc:729] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c69b1f11c3f1451585873b6f5e1b76ee, State: Initialized, Role: FOLLOWER
I20260809 19:20:46.572260 4167 consensus_queue.cc:260] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69b1f11c3f1451585873b6f5e1b76ee" member_type: VOTER last_known_addr { host: "127.3.241.254" port: 33225 } }
I20260809 19:20:46.572348 4167 raft_consensus.cc:388] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:20:46.572397 4167 raft_consensus.cc:482] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:20:46.572453 4167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:46.573140 4167 raft_consensus.cc:504] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69b1f11c3f1451585873b6f5e1b76ee" member_type: VOTER last_known_addr { host: "127.3.241.254" port: 33225 } }
I20260809 19:20:46.573256 4167 leader_election.cc:302] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c69b1f11c3f1451585873b6f5e1b76ee; no voters:
I20260809 19:20:46.573422 4167 leader_election.cc:288] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:20:46.573477 4183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:20:46.573729 4183 raft_consensus.cc:686] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [term 1 LEADER]: Becoming Leader. State: Replica: c69b1f11c3f1451585873b6f5e1b76ee, State: Running, Role: LEADER
I20260809 19:20:46.573737 4167 sys_catalog.cc:564] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:20:46.573933 4183 consensus_queue.cc:237] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69b1f11c3f1451585873b6f5e1b76ee" member_type: VOTER last_known_addr { host: "127.3.241.254" port: 33225 } }
I20260809 19:20:46.574340 4185 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c69b1f11c3f1451585873b6f5e1b76ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69b1f11c3f1451585873b6f5e1b76ee" member_type: VOTER last_known_addr { host: "127.3.241.254" port: 33225 } } }
I20260809 19:20:46.574548 4185 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [sys.catalog]: This master's current role is: LEADER
I20260809 19:20:46.575409 4186 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader c69b1f11c3f1451585873b6f5e1b76ee. Latest consensus state: current_term: 1 leader_uuid: "c69b1f11c3f1451585873b6f5e1b76ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69b1f11c3f1451585873b6f5e1b76ee" member_type: VOTER last_known_addr { host: "127.3.241.254" port: 33225 } } }
I20260809 19:20:46.575632 4186 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee [sys.catalog]: This master's current role is: LEADER
W20260809 19:20:46.575968 4213 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:20:46.576035 4213 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:20:46.576143 4201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:20:46.576433 4201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:20:46.578085 4201 catalog_manager.cc:1347] Generated new cluster ID: 1fe5cdc3e67945c89ab2577663b6b9d3
I20260809 19:20:46.578140 4201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:20:46.604961 4201 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:20:46.605615 4201 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:20:46.609679 4201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c69b1f11c3f1451585873b6f5e1b76ee: Generated new TSK 0
I20260809 19:20:46.609884 4201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:20:46.621425 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.193:0
--local_ip_for_outbound_sockets=127.3.241.193
--webserver_interface=127.3.241.193
--webserver_port=0
--tserver_master_addrs=127.3.241.254:33225
--builtin_ntp_servers=127.3.241.212:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
W20260809 19:20:46.732509 4217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:46.732718 4217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:46.732740 4217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:46.733985 4217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:46.734041 4217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.193
I20260809 19:20:46.735416 4217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.3.241.193
--webserver_port=0
--tserver_master_addrs=127.3.241.254:33225
--never_fsync=true
--heap_profile_path=/tmp/kudu.4217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.193
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:46.735674 4217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:46.736076 4217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:46.742440 4223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:46.742406 4228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:46.742414 4225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:46.743211 4217 server_base.cc:1034] running on GCE node
I20260809 19:20:46.743438 4217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:46.743652 4217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:46.744848 4217 hybrid_clock.cc:648] HybridClock initialized: now 1786303246744841 us; error 35 us; skew 500 ppm
I20260809 19:20:46.746469 4217 webserver.cc:460] Webserver started at http://127.3.241.193:33017/ using document root <none> and password file <none>
I20260809 19:20:46.746716 4217 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:46.746820 4217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:46.746978 4217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:46.748283 4217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "83bb82c10d204b4fa88cb3c709c072ee"
format_stamp: "Formatted at 2026-08-09 19:20:46 on dist-test-slave-gmjp"
I20260809 19:20:46.748675 4217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "83bb82c10d204b4fa88cb3c709c072ee"
format_stamp: "Formatted at 2026-08-09 19:20:46 on dist-test-slave-gmjp"
I20260809 19:20:46.752413 4217 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260809 19:20:46.753445 4235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:46.753671 4217 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:46.753791 4217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "83bb82c10d204b4fa88cb3c709c072ee"
format_stamp: "Formatted at 2026-08-09 19:20:46 on dist-test-slave-gmjp"
I20260809 19:20:46.753878 4217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:46.767247 4217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:46.767619 4217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:46.767786 4217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:46.768069 4217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:46.768471 4217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:46.768507 4217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:46.768550 4217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:46.768579 4217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:46.776741 4217 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.193:34037
I20260809 19:20:46.776873 4429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.193:34037 every 8 connection(s)
I20260809 19:20:46.777173 4217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260809 19:20:46.778218 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 4217
I20260809 19:20:46.778333 4039 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260809 19:20:46.779500 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.194:0
--local_ip_for_outbound_sockets=127.3.241.194
--webserver_interface=127.3.241.194
--webserver_port=0
--tserver_master_addrs=127.3.241.254:33225
--builtin_ntp_servers=127.3.241.212:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260809 19:20:46.784514 4431 heartbeater.cc:347] Connected to a master server at 127.3.241.254:33225
I20260809 19:20:46.784636 4431 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:46.784849 4431 heartbeater.cc:511] Master 127.3.241.254:33225 requested a full tablet report, sending...
I20260809 19:20:46.785511 4108 ts_manager.cc:194] Registered new tserver with Master: 83bb82c10d204b4fa88cb3c709c072ee (127.3.241.193:34037)
I20260809 19:20:46.786186 4108 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.193:59655
W20260809 19:20:46.894546 4435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:46.894797 4435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:46.894835 4435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:46.895979 4435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:46.896057 4435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.194
I20260809 19:20:46.898033 4435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.3.241.194
--webserver_port=0
--tserver_master_addrs=127.3.241.254:33225
--never_fsync=true
--heap_profile_path=/tmp/kudu.4435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.194
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:46.898317 4435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:46.898649 4435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:46.903960 4441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:46.903960 4445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:46.903986 4442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:46.904280 4435 server_base.cc:1034] running on GCE node
I20260809 19:20:46.904649 4435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:46.904981 4435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:46.906203 4435 hybrid_clock.cc:648] HybridClock initialized: now 1786303246906176 us; error 42 us; skew 500 ppm
I20260809 19:20:46.907918 4435 webserver.cc:460] Webserver started at http://127.3.241.194:37147/ using document root <none> and password file <none>
I20260809 19:20:46.908277 4435 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:46.908390 4435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:46.908569 4435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:46.909660 4435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "358adb4de8174e02a1b793645e743177"
format_stamp: "Formatted at 2026-08-09 19:20:46 on dist-test-slave-gmjp"
I20260809 19:20:46.910061 4435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "358adb4de8174e02a1b793645e743177"
format_stamp: "Formatted at 2026-08-09 19:20:46 on dist-test-slave-gmjp"
I20260809 19:20:46.911610 4435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:20:46.912482 4454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:46.912657 4435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:46.912767 4435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "358adb4de8174e02a1b793645e743177"
format_stamp: "Formatted at 2026-08-09 19:20:46 on dist-test-slave-gmjp"
I20260809 19:20:46.912843 4435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:46.925037 4435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:46.925340 4435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:46.925505 4435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:46.925737 4435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:46.926127 4435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:46.926191 4435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:46.926234 4435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:46.926260 4435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:46.934526 4435 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.194:40425
I20260809 19:20:46.934571 4637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.194:40425 every 8 connection(s)
I20260809 19:20:46.934957 4435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260809 19:20:46.937294 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 4435
I20260809 19:20:46.937366 4039 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260809 19:20:46.938611 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.195:0
--local_ip_for_outbound_sockets=127.3.241.195
--webserver_interface=127.3.241.195
--webserver_port=0
--tserver_master_addrs=127.3.241.254:33225
--builtin_ntp_servers=127.3.241.212:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260809 19:20:46.942399 4639 heartbeater.cc:347] Connected to a master server at 127.3.241.254:33225
I20260809 19:20:46.942487 4639 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:46.942735 4639 heartbeater.cc:511] Master 127.3.241.254:33225 requested a full tablet report, sending...
I20260809 19:20:46.943169 4108 ts_manager.cc:194] Registered new tserver with Master: 358adb4de8174e02a1b793645e743177 (127.3.241.194:40425)
I20260809 19:20:46.943682 4108 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.194:47521
W20260809 19:20:47.026052 4644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:47.026304 4644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:47.026346 4644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:47.027490 4644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:47.027541 4644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.195
I20260809 19:20:47.028975 4644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.3.241.195
--webserver_port=0
--tserver_master_addrs=127.3.241.254:33225
--never_fsync=true
--heap_profile_path=/tmp/kudu.4644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.195
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:47.029228 4644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:47.029546 4644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:47.035203 4659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:47.035212 4652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:47.035221 4655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:47.035508 4644 server_base.cc:1034] running on GCE node
I20260809 19:20:47.035799 4644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:47.036118 4644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:47.037323 4644 hybrid_clock.cc:648] HybridClock initialized: now 1786303247037313 us; error 28 us; skew 500 ppm
I20260809 19:20:47.038691 4644 webserver.cc:460] Webserver started at http://127.3.241.195:39281/ using document root <none> and password file <none>
I20260809 19:20:47.038956 4644 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:47.039028 4644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:47.039177 4644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:47.040338 4644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "f29e0aa5b524470892cb3b6dcc16fd7e"
format_stamp: "Formatted at 2026-08-09 19:20:47 on dist-test-slave-gmjp"
I20260809 19:20:47.040750 4644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "f29e0aa5b524470892cb3b6dcc16fd7e"
format_stamp: "Formatted at 2026-08-09 19:20:47 on dist-test-slave-gmjp"
I20260809 19:20:47.042078 4644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:20:47.042840 4672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:47.043032 4644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:20:47.043156 4644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "f29e0aa5b524470892cb3b6dcc16fd7e"
format_stamp: "Formatted at 2026-08-09 19:20:47 on dist-test-slave-gmjp"
I20260809 19:20:47.043231 4644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:47.051970 4644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:47.052320 4644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:47.052472 4644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:47.052779 4644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:47.053196 4644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:47.053257 4644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:47.053305 4644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:47.053332 4644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:47.064007 4644 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.195:46221
I20260809 19:20:47.064046 4839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.195:46221 every 8 connection(s)
I20260809 19:20:47.064524 4644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260809 19:20:47.066468 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 4644
I20260809 19:20:47.066555 4039 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260809 19:20:47.070888 4840 heartbeater.cc:347] Connected to a master server at 127.3.241.254:33225
I20260809 19:20:47.070977 4840 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:47.071130 4840 heartbeater.cc:511] Master 127.3.241.254:33225 requested a full tablet report, sending...
I20260809 19:20:47.071542 4108 ts_manager.cc:194] Registered new tserver with Master: f29e0aa5b524470892cb3b6dcc16fd7e (127.3.241.195:46221)
I20260809 19:20:47.071918 4108 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.195:47205
I20260809 19:20:47.079051 4039 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:20:47.087586 4108 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48854:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
W20260809 19:20:47.089388 4108 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:20:47.096871 4532 tablet_service.cc:1467] Processing CreateTablet for tablet 9140e2f6bd624ceebd333a056d9c6b7e (DEFAULT_TABLE table=TestTable [id=e4a2ded29c8f46779bb0bb81789ba643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:47.096975 4740 tablet_service.cc:1467] Processing CreateTablet for tablet 9140e2f6bd624ceebd333a056d9c6b7e (DEFAULT_TABLE table=TestTable [id=e4a2ded29c8f46779bb0bb81789ba643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:47.097311 4532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9140e2f6bd624ceebd333a056d9c6b7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:47.097362 4740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9140e2f6bd624ceebd333a056d9c6b7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:47.100384 4864 tablet_bootstrap.cc:492] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177: Bootstrap starting.
I20260809 19:20:47.100551 4866 tablet_bootstrap.cc:492] T 9140e2f6bd624ceebd333a056d9c6b7e P f29e0aa5b524470892cb3b6dcc16fd7e: Bootstrap starting.
I20260809 19:20:47.100708 4317 tablet_service.cc:1467] Processing CreateTablet for tablet 9140e2f6bd624ceebd333a056d9c6b7e (DEFAULT_TABLE table=TestTable [id=e4a2ded29c8f46779bb0bb81789ba643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:47.101235 4317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9140e2f6bd624ceebd333a056d9c6b7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:47.101331 4864 tablet_bootstrap.cc:654] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:47.101456 4866 tablet_bootstrap.cc:654] T 9140e2f6bd624ceebd333a056d9c6b7e P f29e0aa5b524470892cb3b6dcc16fd7e: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:47.101775 4864 log.cc:824] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:47.101795 4866 log.cc:824] T 9140e2f6bd624ceebd333a056d9c6b7e P f29e0aa5b524470892cb3b6dcc16fd7e: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:47.102541 4866 tablet_bootstrap.cc:492] T 9140e2f6bd624ceebd333a056d9c6b7e P f29e0aa5b524470892cb3b6dcc16fd7e: No bootstrap required, opened a new log
I20260809 19:20:47.102607 4864 tablet_bootstrap.cc:492] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177: No bootstrap required, opened a new log
I20260809 19:20:47.102635 4866 ts_tablet_manager.cc:1397] T 9140e2f6bd624ceebd333a056d9c6b7e P f29e0aa5b524470892cb3b6dcc16fd7e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:20:47.102679 4864 ts_tablet_manager.cc:1397] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:20:47.104789 4866 raft_consensus.cc:348] T 9140e2f6bd624ceebd333a056d9c6b7e P f29e0aa5b524470892cb3b6dcc16fd7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bb82c10d204b4fa88cb3c709c072ee" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 34037 } } peers { permanent_uuid: "f29e0aa5b524470892cb3b6dcc16fd7e" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 46221 } } peers { permanent_uuid: "358adb4de8174e02a1b793645e743177" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 40425 } }
I20260809 19:20:47.104959 4866 raft_consensus.cc:374] T 9140e2f6bd624ceebd333a056d9c6b7e P f29e0aa5b524470892cb3b6dcc16fd7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:47.105005 4866 raft_consensus.cc:729] T 9140e2f6bd624ceebd333a056d9c6b7e P f29e0aa5b524470892cb3b6dcc16fd7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f29e0aa5b524470892cb3b6dcc16fd7e, State: Initialized, Role: FOLLOWER
I20260809 19:20:47.105123 4866 consensus_queue.cc:260] T 9140e2f6bd624ceebd333a056d9c6b7e P f29e0aa5b524470892cb3b6dcc16fd7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bb82c10d204b4fa88cb3c709c072ee" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 34037 } } peers { permanent_uuid: "f29e0aa5b524470892cb3b6dcc16fd7e" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 46221 } } peers { permanent_uuid: "358adb4de8174e02a1b793645e743177" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 40425 } }
I20260809 19:20:47.105381 4866 ts_tablet_manager.cc:1428] T 9140e2f6bd624ceebd333a056d9c6b7e P f29e0aa5b524470892cb3b6dcc16fd7e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:20:47.105454 4840 heartbeater.cc:503] Master 127.3.241.254:33225 was elected leader, sending a full tablet report...
I20260809 19:20:47.105581 4864 raft_consensus.cc:348] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bb82c10d204b4fa88cb3c709c072ee" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 34037 } } peers { permanent_uuid: "f29e0aa5b524470892cb3b6dcc16fd7e" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 46221 } } peers { permanent_uuid: "358adb4de8174e02a1b793645e743177" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 40425 } }
I20260809 19:20:47.105772 4864 raft_consensus.cc:374] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:47.105821 4864 raft_consensus.cc:729] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 358adb4de8174e02a1b793645e743177, State: Initialized, Role: FOLLOWER
I20260809 19:20:47.105934 4864 consensus_queue.cc:260] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bb82c10d204b4fa88cb3c709c072ee" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 34037 } } peers { permanent_uuid: "f29e0aa5b524470892cb3b6dcc16fd7e" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 46221 } } peers { permanent_uuid: "358adb4de8174e02a1b793645e743177" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 40425 } }
I20260809 19:20:47.106228 4864 ts_tablet_manager.cc:1428] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 19:20:47.106240 4639 heartbeater.cc:503] Master 127.3.241.254:33225 was elected leader, sending a full tablet report...
I20260809 19:20:47.106470 4876 tablet_bootstrap.cc:492] T 9140e2f6bd624ceebd333a056d9c6b7e P 83bb82c10d204b4fa88cb3c709c072ee: Bootstrap starting.
I20260809 19:20:47.107355 4876 tablet_bootstrap.cc:654] T 9140e2f6bd624ceebd333a056d9c6b7e P 83bb82c10d204b4fa88cb3c709c072ee: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:47.107694 4876 log.cc:824] T 9140e2f6bd624ceebd333a056d9c6b7e P 83bb82c10d204b4fa88cb3c709c072ee: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:47.108538 4876 tablet_bootstrap.cc:492] T 9140e2f6bd624ceebd333a056d9c6b7e P 83bb82c10d204b4fa88cb3c709c072ee: No bootstrap required, opened a new log
I20260809 19:20:47.108626 4876 ts_tablet_manager.cc:1397] T 9140e2f6bd624ceebd333a056d9c6b7e P 83bb82c10d204b4fa88cb3c709c072ee: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:20:47.110522 4876 raft_consensus.cc:348] T 9140e2f6bd624ceebd333a056d9c6b7e P 83bb82c10d204b4fa88cb3c709c072ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bb82c10d204b4fa88cb3c709c072ee" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 34037 } } peers { permanent_uuid: "f29e0aa5b524470892cb3b6dcc16fd7e" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 46221 } } peers { permanent_uuid: "358adb4de8174e02a1b793645e743177" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 40425 } }
I20260809 19:20:47.110682 4876 raft_consensus.cc:374] T 9140e2f6bd624ceebd333a056d9c6b7e P 83bb82c10d204b4fa88cb3c709c072ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:47.110730 4876 raft_consensus.cc:729] T 9140e2f6bd624ceebd333a056d9c6b7e P 83bb82c10d204b4fa88cb3c709c072ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83bb82c10d204b4fa88cb3c709c072ee, State: Initialized, Role: FOLLOWER
I20260809 19:20:47.110875 4876 consensus_queue.cc:260] T 9140e2f6bd624ceebd333a056d9c6b7e P 83bb82c10d204b4fa88cb3c709c072ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bb82c10d204b4fa88cb3c709c072ee" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 34037 } } peers { permanent_uuid: "f29e0aa5b524470892cb3b6dcc16fd7e" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 46221 } } peers { permanent_uuid: "358adb4de8174e02a1b793645e743177" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 40425 } }
I20260809 19:20:47.111219 4876 ts_tablet_manager.cc:1428] T 9140e2f6bd624ceebd333a056d9c6b7e P 83bb82c10d204b4fa88cb3c709c072ee: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:20:47.111286 4431 heartbeater.cc:503] Master 127.3.241.254:33225 was elected leader, sending a full tablet report...
I20260809 19:20:47.174606 4881 raft_consensus.cc:482] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:20:47.174762 4881 raft_consensus.cc:504] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bb82c10d204b4fa88cb3c709c072ee" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 34037 } } peers { permanent_uuid: "f29e0aa5b524470892cb3b6dcc16fd7e" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 46221 } } peers { permanent_uuid: "358adb4de8174e02a1b793645e743177" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 40425 } }
I20260809 19:20:47.175184 4881 leader_election.cc:288] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83bb82c10d204b4fa88cb3c709c072ee (127.3.241.193:34037), f29e0aa5b524470892cb3b6dcc16fd7e (127.3.241.195:46221)
I20260809 19:20:47.178386 4775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9140e2f6bd624ceebd333a056d9c6b7e" candidate_uuid: "358adb4de8174e02a1b793645e743177" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f29e0aa5b524470892cb3b6dcc16fd7e" is_pre_election: true
I20260809 19:20:47.178603 4775 raft_consensus.cc:2454] T 9140e2f6bd624ceebd333a056d9c6b7e P f29e0aa5b524470892cb3b6dcc16fd7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 358adb4de8174e02a1b793645e743177 in term 0.
I20260809 19:20:47.178596 4357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9140e2f6bd624ceebd333a056d9c6b7e" candidate_uuid: "358adb4de8174e02a1b793645e743177" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83bb82c10d204b4fa88cb3c709c072ee" is_pre_election: true
I20260809 19:20:47.178742 4357 raft_consensus.cc:2454] T 9140e2f6bd624ceebd333a056d9c6b7e P 83bb82c10d204b4fa88cb3c709c072ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 358adb4de8174e02a1b793645e743177 in term 0.
I20260809 19:20:47.178995 4462 leader_election.cc:302] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 358adb4de8174e02a1b793645e743177, 83bb82c10d204b4fa88cb3c709c072ee; no voters:
I20260809 19:20:47.179219 4881 raft_consensus.cc:2781] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:20:47.179298 4881 raft_consensus.cc:482] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:20:47.179330 4881 raft_consensus.cc:3037] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:47.180049 4881 raft_consensus.cc:504] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bb82c10d204b4fa88cb3c709c072ee" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 34037 } } peers { permanent_uuid: "f29e0aa5b524470892cb3b6dcc16fd7e" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 46221 } } peers { permanent_uuid: "358adb4de8174e02a1b793645e743177" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 40425 } }
I20260809 19:20:47.180256 4881 leader_election.cc:288] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [CANDIDATE]: Term 1 election: Requested vote from peers 83bb82c10d204b4fa88cb3c709c072ee (127.3.241.193:34037), f29e0aa5b524470892cb3b6dcc16fd7e (127.3.241.195:46221)
I20260809 19:20:47.180436 4357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9140e2f6bd624ceebd333a056d9c6b7e" candidate_uuid: "358adb4de8174e02a1b793645e743177" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83bb82c10d204b4fa88cb3c709c072ee"
I20260809 19:20:47.180526 4357 raft_consensus.cc:3037] T 9140e2f6bd624ceebd333a056d9c6b7e P 83bb82c10d204b4fa88cb3c709c072ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:47.180572 4775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9140e2f6bd624ceebd333a056d9c6b7e" candidate_uuid: "358adb4de8174e02a1b793645e743177" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f29e0aa5b524470892cb3b6dcc16fd7e"
I20260809 19:20:47.180670 4775 raft_consensus.cc:3037] T 9140e2f6bd624ceebd333a056d9c6b7e P f29e0aa5b524470892cb3b6dcc16fd7e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:47.181478 4357 raft_consensus.cc:2454] T 9140e2f6bd624ceebd333a056d9c6b7e P 83bb82c10d204b4fa88cb3c709c072ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 358adb4de8174e02a1b793645e743177 in term 1.
I20260809 19:20:47.181622 4775 raft_consensus.cc:2454] T 9140e2f6bd624ceebd333a056d9c6b7e P f29e0aa5b524470892cb3b6dcc16fd7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 358adb4de8174e02a1b793645e743177 in term 1.
I20260809 19:20:47.181697 4462 leader_election.cc:302] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 358adb4de8174e02a1b793645e743177, 83bb82c10d204b4fa88cb3c709c072ee; no voters:
I20260809 19:20:47.181790 4881 raft_consensus.cc:2781] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:20:47.181953 4881 raft_consensus.cc:686] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [term 1 LEADER]: Becoming Leader. State: Replica: 358adb4de8174e02a1b793645e743177, State: Running, Role: LEADER
I20260809 19:20:47.182057 4881 consensus_queue.cc:237] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bb82c10d204b4fa88cb3c709c072ee" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 34037 } } peers { permanent_uuid: "f29e0aa5b524470892cb3b6dcc16fd7e" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 46221 } } peers { permanent_uuid: "358adb4de8174e02a1b793645e743177" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 40425 } }
I20260809 19:20:47.182909 4107 catalog_manager.cc:5495] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 reported cstate change: term changed from 0 to 1, leader changed from <none> to 358adb4de8174e02a1b793645e743177 (127.3.241.194). New cstate: current_term: 1 leader_uuid: "358adb4de8174e02a1b793645e743177" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83bb82c10d204b4fa88cb3c709c072ee" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 34037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f29e0aa5b524470892cb3b6dcc16fd7e" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 46221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "358adb4de8174e02a1b793645e743177" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 40425 } health_report { overall_health: HEALTHY } } }
W20260809 19:20:47.185668 4640 tablet.cc:2369] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:20:47.209491 4039 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:20:47.210168 4039 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 83bb82c10d204b4fa88cb3c709c072ee to finish bootstrapping
I20260809 19:20:47.214092 4039 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 358adb4de8174e02a1b793645e743177 to finish bootstrapping
I20260809 19:20:47.217521 4039 ts_itest-base.cc:233] Waiting for 1 tablets on tserver f29e0aa5b524470892cb3b6dcc16fd7e to finish bootstrapping
I20260809 19:20:47.230299 4039 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0 0.0
W20260809 19:20:47.277983 4432 tablet.cc:2369] T 9140e2f6bd624ceebd333a056d9c6b7e P 83bb82c10d204b4fa88cb3c709c072ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:20:47.315346 4843 tablet.cc:2369] T 9140e2f6bd624ceebd333a056d9c6b7e P f29e0aa5b524470892cb3b6dcc16fd7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:20:47.331970 4039 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0 0.0
I20260809 19:20:47.533749 4039 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0 0.0
I20260809 19:20:47.679119 4881 consensus_queue.cc:1035] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83bb82c10d204b4fa88cb3c709c072ee" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 34037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:47.681603 4902 consensus_queue.cc:1035] T 9140e2f6bd624ceebd333a056d9c6b7e P 358adb4de8174e02a1b793645e743177 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f29e0aa5b524470892cb3b6dcc16fd7e" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 46221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 19:20:47.837797 4901 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:54266: Not implemented: Unknown operation type: 115
I20260809 19:20:47.870432 4532 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260809 19:20:47.870755 4740 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260809 19:20:47.872359 4317 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
c69b1f11c3f1451585873b6f5e1b76ee | 127.3.241.254:33225 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.3.241.212:44059 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
358adb4de8174e02a1b793645e743177 | 127.3.241.194:40425 | HEALTHY | <none> | 1 | 0
83bb82c10d204b4fa88cb3c709c072ee | 127.3.241.193:34037 | HEALTHY | <none> | 0 | 0
f29e0aa5b524470892cb3b6dcc16fd7e | 127.3.241.195:46221 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
local_ip_for_outbound_sockets | 127.3.241.193 | experimental | 127.3.241.193:34037
local_ip_for_outbound_sockets | 127.3.241.194 | experimental | 127.3.241.194:40425
local_ip_for_outbound_sockets | 127.3.241.195 | experimental | 127.3.241.195:46221
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.3.241.193:34037
server_dump_info_path | /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.3.241.194:40425
server_dump_info_path | /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.3.241.195:46221
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.3.241.212:44059 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260809 19:20:47.880525 4039 log_verifier.cc:126] Checking tablet 9140e2f6bd624ceebd333a056d9c6b7e
I20260809 19:20:47.901232 4039 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 9140e2f6bd624ceebd333a056d9c6b7e
I20260809 19:20:48.232546 4039 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKf1XAf/build/release/bin/kudu with pid 4217
I20260809 19:20:48.247006 4039 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKf1XAf/build/release/bin/kudu with pid 4435
I20260809 19:20:48.256865 4039 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKf1XAf/build/release/bin/kudu with pid 4644
I20260809 19:20:48.267035 4039 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKf1XAf/build/release/bin/kudu with pid 4058
2026-08-09T19:20:48Z chronyd exiting
[ OK ] RaftConsensusITest.TestFailedOp (1875 ms)
[ RUN ] RaftConsensusITest.MultiThreadedInsertWithFailovers
I20260809 19:20:48.295017 4039 test_util.cc:268] Using random seed: 580084835
I20260809 19:20:48.295567 4039 ts_itest-base.cc:112] Starting cluster with:
I20260809 19:20:48.295601 4039 ts_itest-base.cc:113] --------------
I20260809 19:20:48.295614 4039 ts_itest-base.cc:114] 7 tablet servers
I20260809 19:20:48.295624 4039 ts_itest-base.cc:115] 7 replicas per TS
I20260809 19:20:48.295635 4039 ts_itest-base.cc:116] --------------
2026-08-09T19:20:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:20:48Z Disabled control of system clock
I20260809 19:20:48.304729 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.241.254:44125
--webserver_interface=127.3.241.254
--webserver_port=0
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.241.254:44125 with env {}
W20260809 19:20:48.386525 4988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:48.386737 4988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:48.386757 4988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:48.387814 4988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:20:48.387848 4988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:48.387862 4988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:20:48.387874 4988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:20:48.389163 4988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.241.254:44125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.241.254:44125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.3.241.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.4988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:48.389364 4988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:48.389609 4988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:48.394450 5004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:48.394443 5008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:48.394466 5002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:48.394618 4988 server_base.cc:1034] running on GCE node
I20260809 19:20:48.394951 4988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:48.395316 4988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:48.396474 4988 hybrid_clock.cc:648] HybridClock initialized: now 1786303248396447 us; error 44 us; skew 500 ppm
I20260809 19:20:48.397672 4988 webserver.cc:460] Webserver started at http://127.3.241.254:37849/ using document root <none> and password file <none>
I20260809 19:20:48.397935 4988 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:48.397984 4988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:48.398138 4988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:48.399159 4988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "5a1ebab4acf543e79152348ec3bfef88"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:48.399538 4988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "5a1ebab4acf543e79152348ec3bfef88"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:48.401059 4988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:20:48.401875 5017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:48.402175 4988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:20:48.402273 4988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/wal
uuid: "5a1ebab4acf543e79152348ec3bfef88"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:48.402354 4988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:48.417514 4988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:48.417843 4988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:48.418004 4988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:48.422302 4988 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.254:44125
I20260809 19:20:48.422363 5103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.254:44125 every 8 connection(s)
I20260809 19:20:48.422724 4988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260809 19:20:48.423405 5106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:48.426117 5106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88: Bootstrap starting.
I20260809 19:20:48.426700 5106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:48.426950 5106 log.cc:824] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:48.427606 5106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88: No bootstrap required, opened a new log
I20260809 19:20:48.429327 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 4988
I20260809 19:20:48.429419 4039 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260809 19:20:48.429446 5106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1ebab4acf543e79152348ec3bfef88" member_type: VOTER last_known_addr { host: "127.3.241.254" port: 44125 } }
I20260809 19:20:48.429627 5106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:48.429665 5106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a1ebab4acf543e79152348ec3bfef88, State: Initialized, Role: FOLLOWER
I20260809 19:20:48.429826 5106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1ebab4acf543e79152348ec3bfef88" member_type: VOTER last_known_addr { host: "127.3.241.254" port: 44125 } }
I20260809 19:20:48.429931 5106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:20:48.429966 5106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:20:48.430019 5106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:48.430724 5106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1ebab4acf543e79152348ec3bfef88" member_type: VOTER last_known_addr { host: "127.3.241.254" port: 44125 } }
I20260809 19:20:48.430856 5106 leader_election.cc:302] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a1ebab4acf543e79152348ec3bfef88; no voters:
I20260809 19:20:48.431033 5106 leader_election.cc:288] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:20:48.431087 5116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:20:48.431224 5116 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [term 1 LEADER]: Becoming Leader. State: Replica: 5a1ebab4acf543e79152348ec3bfef88, State: Running, Role: LEADER
I20260809 19:20:48.431310 5106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:20:48.431313 5116 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1ebab4acf543e79152348ec3bfef88" member_type: VOTER last_known_addr { host: "127.3.241.254" port: 44125 } }
I20260809 19:20:48.431667 5116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a1ebab4acf543e79152348ec3bfef88. Latest consensus state: current_term: 1 leader_uuid: "5a1ebab4acf543e79152348ec3bfef88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1ebab4acf543e79152348ec3bfef88" member_type: VOTER last_known_addr { host: "127.3.241.254" port: 44125 } } }
I20260809 19:20:48.431738 5116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [sys.catalog]: This master's current role is: LEADER
I20260809 19:20:48.431941 5126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:20:48.432317 5117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a1ebab4acf543e79152348ec3bfef88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a1ebab4acf543e79152348ec3bfef88" member_type: VOTER last_known_addr { host: "127.3.241.254" port: 44125 } } }
I20260809 19:20:48.432487 5117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88 [sys.catalog]: This master's current role is: LEADER
I20260809 19:20:48.432464 5126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:20:48.434228 5126 catalog_manager.cc:1347] Generated new cluster ID: d2874399323c4cac8df9b34bd003c1a6
I20260809 19:20:48.434284 5126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:20:48.441959 5126 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:20:48.442631 5126 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:20:48.446810 5126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a1ebab4acf543e79152348ec3bfef88: Generated new TSK 0
I20260809 19:20:48.446986 5126 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:20:48.449805 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.193:0
--local_ip_for_outbound_sockets=127.3.241.193
--webserver_interface=127.3.241.193
--webserver_port=0
--tserver_master_addrs=127.3.241.254:44125
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260809 19:20:48.536239 5148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:48.536481 5148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:48.536518 5148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:48.537715 5148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:48.537798 5148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.193
I20260809 19:20:48.539201 5148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.3.241.193
--webserver_port=0
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.5148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.193
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:48.539492 5148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:48.539826 5148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:48.544962 5161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:48.545025 5155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:48.544953 5157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:48.545668 5148 server_base.cc:1034] running on GCE node
I20260809 19:20:48.545899 5148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:48.546185 5148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:48.547379 5148 hybrid_clock.cc:648] HybridClock initialized: now 1786303248547357 us; error 33 us; skew 500 ppm
I20260809 19:20:48.548955 5148 webserver.cc:460] Webserver started at http://127.3.241.193:33375/ using document root <none> and password file <none>
I20260809 19:20:48.549278 5148 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:48.549350 5148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:48.549489 5148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:48.550863 5148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "dd68273a4b314386ae0f5489e2f1d468"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:48.551265 5148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "dd68273a4b314386ae0f5489e2f1d468"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:48.553107 5148 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 19:20:48.553970 5170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:48.554176 5148 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:48.554277 5148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "dd68273a4b314386ae0f5489e2f1d468"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:48.554349 5148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:48.572643 5148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:48.572981 5148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:48.573148 5148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:48.573375 5148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:48.573724 5148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:48.573782 5148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:48.573823 5148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:48.573853 5148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:48.579946 5148 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.193:46757
I20260809 19:20:48.580044 5347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.193:46757 every 8 connection(s)
I20260809 19:20:48.580402 5148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260809 19:20:48.585793 5348 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:20:48.585891 5348 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:48.585878 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 5148
I20260809 19:20:48.585987 4039 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260809 19:20:48.586100 5348 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:20:48.586666 5046 ts_manager.cc:194] Registered new tserver with Master: dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757)
I20260809 19:20:48.587210 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.194:0
--local_ip_for_outbound_sockets=127.3.241.194
--webserver_interface=127.3.241.194
--webserver_port=0
--tserver_master_addrs=127.3.241.254:44125
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260809 19:20:48.587374 5046 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.193:59345
W20260809 19:20:48.671114 5353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:48.671322 5353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:48.671343 5353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:48.672612 5353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:48.672667 5353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.194
I20260809 19:20:48.674002 5353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.3.241.194
--webserver_port=0
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.5353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.194
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:48.674237 5353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:48.674546 5353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:48.679581 5360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:48.679611 5353 server_base.cc:1034] running on GCE node
W20260809 19:20:48.679580 5363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:48.679610 5361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:48.680071 5353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:48.680369 5353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:48.681531 5353 hybrid_clock.cc:648] HybridClock initialized: now 1786303248681555 us; error 62 us; skew 500 ppm
I20260809 19:20:48.682871 5353 webserver.cc:460] Webserver started at http://127.3.241.194:35399/ using document root <none> and password file <none>
I20260809 19:20:48.683148 5353 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:48.683195 5353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:48.683358 5353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:48.684352 5353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "7d4177500c7a4f41afce5190ff786506"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:48.684701 5353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "7d4177500c7a4f41afce5190ff786506"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:48.686095 5353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:20:48.686818 5373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:48.687032 5353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:48.687122 5353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "7d4177500c7a4f41afce5190ff786506"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:48.687196 5353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:48.702214 5353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:48.702507 5353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:48.702643 5353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:48.702893 5353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:48.703224 5353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:48.703259 5353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:48.703281 5353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:48.703294 5353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:48.709502 5353 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.194:41469
I20260809 19:20:48.709553 5557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.194:41469 every 8 connection(s)
I20260809 19:20:48.709885 5353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260809 19:20:48.712946 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 5353
I20260809 19:20:48.713068 4039 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260809 19:20:48.714316 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.195:0
--local_ip_for_outbound_sockets=127.3.241.195
--webserver_interface=127.3.241.195
--webserver_port=0
--tserver_master_addrs=127.3.241.254:44125
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260809 19:20:48.716384 5558 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:20:48.716485 5558 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:48.716688 5558 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:20:48.717114 5046 ts_manager.cc:194] Registered new tserver with Master: 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469)
I20260809 19:20:48.717507 5046 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.194:36901
W20260809 19:20:48.798942 5565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:48.799162 5565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:48.799199 5565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:48.800411 5565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:48.800493 5565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.195
I20260809 19:20:48.801802 5565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.3.241.195
--webserver_port=0
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.5565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.195
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:48.802058 5565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:48.802358 5565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:48.807698 5574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:48.807688 5572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:48.807688 5576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:48.807929 5565 server_base.cc:1034] running on GCE node
I20260809 19:20:48.808204 5565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:48.808449 5565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:48.809587 5565 hybrid_clock.cc:648] HybridClock initialized: now 1786303248809577 us; error 28 us; skew 500 ppm
I20260809 19:20:48.810853 5565 webserver.cc:460] Webserver started at http://127.3.241.195:42257/ using document root <none> and password file <none>
I20260809 19:20:48.811097 5565 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:48.811144 5565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:48.811291 5565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:48.812501 5565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "61427f2036ac4f96a9655aef910a7d7a"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:48.812893 5565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "61427f2036ac4f96a9655aef910a7d7a"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:48.814335 5565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:20:48.815147 5584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:48.815363 5565 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:48.815456 5565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "61427f2036ac4f96a9655aef910a7d7a"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:48.815552 5565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:48.827842 5565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:48.828251 5565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:48.828413 5565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:48.828673 5565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:48.829032 5565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:48.829092 5565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:48.829133 5565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:48.829160 5565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:48.838177 5565 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.195:36997
I20260809 19:20:48.838217 5774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.195:36997 every 8 connection(s)
I20260809 19:20:48.838634 5565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260809 19:20:48.840816 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 5565
I20260809 19:20:48.840939 4039 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260809 19:20:48.842235 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.196:0
--local_ip_for_outbound_sockets=127.3.241.196
--webserver_interface=127.3.241.196
--webserver_port=0
--tserver_master_addrs=127.3.241.254:44125
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260809 19:20:48.845808 5776 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:20:48.845899 5776 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:48.846138 5776 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:20:48.846535 5046 ts_manager.cc:194] Registered new tserver with Master: 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997)
I20260809 19:20:48.847071 5046 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.195:55763
W20260809 19:20:48.928521 5780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:48.928737 5780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:48.928764 5780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:48.929914 5780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:48.929970 5780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.196
I20260809 19:20:48.931322 5780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.3.241.196
--webserver_port=0
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.5780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.196
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:48.931541 5780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:48.931797 5780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:48.937273 5789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:48.937286 5785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:48.937280 5786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:48.937686 5780 server_base.cc:1034] running on GCE node
I20260809 19:20:48.937918 5780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:48.938169 5780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:48.939327 5780 hybrid_clock.cc:648] HybridClock initialized: now 1786303248939320 us; error 45 us; skew 500 ppm
I20260809 19:20:48.940658 5780 webserver.cc:460] Webserver started at http://127.3.241.196:37977/ using document root <none> and password file <none>
I20260809 19:20:48.940905 5780 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:48.940979 5780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:48.941120 5780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:48.942200 5780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "6c075d51984240f6b5d45d0d25a0d595"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:48.942652 5780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "6c075d51984240f6b5d45d0d25a0d595"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:48.944144 5780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:20:48.944983 5802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:48.945147 5780 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:20:48.945256 5780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "6c075d51984240f6b5d45d0d25a0d595"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:48.945333 5780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:48.953085 5780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:48.953401 5780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:48.953563 5780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:48.953845 5780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:48.954242 5780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:48.954303 5780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:48.954345 5780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:48.954370 5780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:48.964439 5780 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.196:35545
I20260809 19:20:48.964493 5999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.196:35545 every 8 connection(s)
I20260809 19:20:48.964912 5780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20260809 19:20:48.969642 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 5780
I20260809 19:20:48.969717 4039 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20260809 19:20:48.971159 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.197:0
--local_ip_for_outbound_sockets=127.3.241.197
--webserver_interface=127.3.241.197
--webserver_port=0
--tserver_master_addrs=127.3.241.254:44125
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260809 19:20:48.971354 6001 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:20:48.971464 6001 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:48.971675 6001 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:20:48.972102 5046 ts_manager.cc:194] Registered new tserver with Master: 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545)
I20260809 19:20:48.972529 5046 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.196:45673
W20260809 19:20:49.056416 6004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:49.056619 6004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:49.056640 6004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:49.057780 6004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:49.057858 6004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.197
I20260809 19:20:49.059315 6004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.197:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.3.241.197
--webserver_port=0
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.6004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.197
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:49.059593 6004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:49.059911 6004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:49.064882 6017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:49.064909 6014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:49.064894 6019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:49.065055 6004 server_base.cc:1034] running on GCE node
I20260809 19:20:49.065353 6004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:49.065599 6004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:49.066763 6004 hybrid_clock.cc:648] HybridClock initialized: now 1786303249066753 us; error 27 us; skew 500 ppm
I20260809 19:20:49.067986 6004 webserver.cc:460] Webserver started at http://127.3.241.197:38231/ using document root <none> and password file <none>
I20260809 19:20:49.068315 6004 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:49.068388 6004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:49.068531 6004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:49.069581 6004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "757ec8723e924451a89abb5cc62c1073"
format_stamp: "Formatted at 2026-08-09 19:20:49 on dist-test-slave-gmjp"
I20260809 19:20:49.070008 6004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "757ec8723e924451a89abb5cc62c1073"
format_stamp: "Formatted at 2026-08-09 19:20:49 on dist-test-slave-gmjp"
I20260809 19:20:49.071415 6004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:20:49.072201 6033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:49.072454 6004 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:49.072571 6004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "757ec8723e924451a89abb5cc62c1073"
format_stamp: "Formatted at 2026-08-09 19:20:49 on dist-test-slave-gmjp"
I20260809 19:20:49.072649 6004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:49.095156 6004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:49.095606 6004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:49.095782 6004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:49.096052 6004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:49.096508 6004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:49.096546 6004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:49.096570 6004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:49.096622 6004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:49.106670 6004 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.197:45111
I20260809 19:20:49.106763 6235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.197:45111 every 8 connection(s)
I20260809 19:20:49.107131 6004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20260809 19:20:49.107462 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 6004
I20260809 19:20:49.107563 4039 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20260809 19:20:49.108949 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.198:0
--local_ip_for_outbound_sockets=127.3.241.198
--webserver_interface=127.3.241.198
--webserver_port=0
--tserver_master_addrs=127.3.241.254:44125
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260809 19:20:49.114027 6236 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:20:49.114143 6236 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:49.114331 6236 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:20:49.114786 5046 ts_manager.cc:194] Registered new tserver with Master: 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111)
I20260809 19:20:49.115175 5046 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.197:49993
W20260809 19:20:49.200632 6242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:49.200865 6242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:49.200899 6242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:49.202021 6242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:49.202099 6242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.198
I20260809 19:20:49.203403 6242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.198:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.3.241.198
--webserver_port=0
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.6242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.198
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:49.203658 6242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:49.203950 6242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:49.209246 6249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:49.209314 6252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:49.209479 6242 server_base.cc:1034] running on GCE node
W20260809 19:20:49.209324 6250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:49.209810 6242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:49.210068 6242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:49.211264 6242 hybrid_clock.cc:648] HybridClock initialized: now 1786303249211251 us; error 27 us; skew 500 ppm
I20260809 19:20:49.215348 6242 webserver.cc:460] Webserver started at http://127.3.241.198:45327/ using document root <none> and password file <none>
I20260809 19:20:49.215590 6242 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:49.215662 6242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:49.215878 6242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:49.216959 6242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
format_stamp: "Formatted at 2026-08-09 19:20:49 on dist-test-slave-gmjp"
I20260809 19:20:49.217334 6242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
format_stamp: "Formatted at 2026-08-09 19:20:49 on dist-test-slave-gmjp"
I20260809 19:20:49.218921 6242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 19:20:49.219810 6263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:49.219996 6242 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:49.220112 6242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
format_stamp: "Formatted at 2026-08-09 19:20:49 on dist-test-slave-gmjp"
I20260809 19:20:49.220222 6242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:49.241278 6242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:49.241596 6242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:49.241752 6242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:49.242022 6242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:49.242369 6242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:49.242427 6242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:49.242475 6242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:49.242499 6242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:49.251464 6242 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.198:39597
I20260809 19:20:49.251506 6456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.198:39597 every 8 connection(s)
I20260809 19:20:49.251902 6242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20260809 19:20:49.256357 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 6242
I20260809 19:20:49.256457 4039 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20260809 19:20:49.257808 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.199:0
--local_ip_for_outbound_sockets=127.3.241.199
--webserver_interface=127.3.241.199
--webserver_port=0
--tserver_master_addrs=127.3.241.254:44125
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260809 19:20:49.258075 6457 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:20:49.258183 6457 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:49.258419 6457 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:20:49.258884 5046 ts_manager.cc:194] Registered new tserver with Master: d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:49.259467 5046 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.198:50137
W20260809 19:20:49.345633 6462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:49.345865 6462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:49.345901 6462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:49.347047 6462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:49.347128 6462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.199
I20260809 19:20:49.348565 6462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.199:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.3.241.199
--webserver_port=0
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.6462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.199
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:49.348819 6462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:49.349107 6462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:49.354211 6468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:49.354254 6477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:49.354221 6472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:49.354377 6462 server_base.cc:1034] running on GCE node
I20260809 19:20:49.354667 6462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:49.354871 6462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:49.356009 6462 hybrid_clock.cc:648] HybridClock initialized: now 1786303249355991 us; error 33 us; skew 500 ppm
I20260809 19:20:49.357372 6462 webserver.cc:460] Webserver started at http://127.3.241.199:44661/ using document root <none> and password file <none>
I20260809 19:20:49.357618 6462 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:49.357666 6462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:49.357838 6462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:49.358978 6462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "23183da38752444a9ff5525be50cd404"
format_stamp: "Formatted at 2026-08-09 19:20:49 on dist-test-slave-gmjp"
I20260809 19:20:49.359421 6462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "23183da38752444a9ff5525be50cd404"
format_stamp: "Formatted at 2026-08-09 19:20:49 on dist-test-slave-gmjp"
I20260809 19:20:49.361333 6462 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 19:20:49.362440 6483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:49.362628 6462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:20:49.362725 6462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "23183da38752444a9ff5525be50cd404"
format_stamp: "Formatted at 2026-08-09 19:20:49 on dist-test-slave-gmjp"
I20260809 19:20:49.362792 6462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:49.384109 6462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:49.384538 6462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:49.384707 6462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:49.384994 6462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:49.385375 6462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:49.385434 6462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:49.385468 6462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:49.385512 6462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:49.395658 6462 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.199:41771
I20260809 19:20:49.395793 6677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.199:41771 every 8 connection(s)
I20260809 19:20:49.396129 6462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20260809 19:20:49.402252 6678 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:20:49.402364 6678 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:49.402521 6678 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:20:49.402905 5046 ts_manager.cc:194] Registered new tserver with Master: 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771)
I20260809 19:20:49.403429 5046 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.199:45589
I20260809 19:20:49.405025 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 6462
I20260809 19:20:49.405125 4039 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20260809 19:20:49.406417 4039 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20260809 19:20:49.413452 5046 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59248:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
W20260809 19:20:49.415095 5046 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20260809 19:20:49.424088 5241 tablet_service.cc:1467] Processing CreateTablet for tablet 7c092d80045840deac3151a94e01de0b (DEFAULT_TABLE table=TestTable [id=d57d7e84cd6e47c18a228a7d20a7f73a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:49.424535 5241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c092d80045840deac3151a94e01de0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:49.425539 5664 tablet_service.cc:1467] Processing CreateTablet for tablet 7c092d80045840deac3151a94e01de0b (DEFAULT_TABLE table=TestTable [id=d57d7e84cd6e47c18a228a7d20a7f73a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:49.425899 5664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c092d80045840deac3151a94e01de0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:49.428500 6117 tablet_service.cc:1467] Processing CreateTablet for tablet 7c092d80045840deac3151a94e01de0b (DEFAULT_TABLE table=TestTable [id=d57d7e84cd6e47c18a228a7d20a7f73a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:49.428836 6117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c092d80045840deac3151a94e01de0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:49.429351 6714 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Bootstrap starting.
I20260809 19:20:49.429849 5451 tablet_service.cc:1467] Processing CreateTablet for tablet 7c092d80045840deac3151a94e01de0b (DEFAULT_TABLE table=TestTable [id=d57d7e84cd6e47c18a228a7d20a7f73a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:49.430146 5451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c092d80045840deac3151a94e01de0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:49.430544 6714 tablet_bootstrap.cc:654] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:49.430891 6714 log.cc:824] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:49.432204 6714 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: No bootstrap required, opened a new log
I20260809 19:20:49.432274 6714 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:20:49.433029 5887 tablet_service.cc:1467] Processing CreateTablet for tablet 7c092d80045840deac3151a94e01de0b (DEFAULT_TABLE table=TestTable [id=d57d7e84cd6e47c18a228a7d20a7f73a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:49.433372 5887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c092d80045840deac3151a94e01de0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:49.433926 6718 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: Bootstrap starting.
I20260809 19:20:49.434934 6718 tablet_bootstrap.cc:654] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:49.435084 6560 tablet_service.cc:1467] Processing CreateTablet for tablet 7c092d80045840deac3151a94e01de0b (DEFAULT_TABLE table=TestTable [id=d57d7e84cd6e47c18a228a7d20a7f73a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:49.435452 6560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c092d80045840deac3151a94e01de0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:49.435484 6718 log.cc:824] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:49.436441 6349 tablet_service.cc:1467] Processing CreateTablet for tablet 7c092d80045840deac3151a94e01de0b (DEFAULT_TABLE table=TestTable [id=d57d7e84cd6e47c18a228a7d20a7f73a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:49.436756 6349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c092d80045840deac3151a94e01de0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:49.437309 6723 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595: Bootstrap starting.
I20260809 19:20:49.438081 6723 tablet_bootstrap.cc:654] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:49.438395 6723 log.cc:824] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:49.439035 6726 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Bootstrap starting.
I20260809 19:20:49.439349 6723 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595: No bootstrap required, opened a new log
I20260809 19:20:49.439419 6723 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:20:49.439702 6726 tablet_bootstrap.cc:654] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:49.439694 6714 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.439877 6714 raft_consensus.cc:374] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:49.439911 6714 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61427f2036ac4f96a9655aef910a7d7a, State: Initialized, Role: FOLLOWER
I20260809 19:20:49.440037 6714 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.440169 6726 log.cc:824] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:49.440421 6714 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Time spent starting tablet: real 0.008s user 0.000s sys 0.004s
I20260809 19:20:49.440579 5776 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
I20260809 19:20:49.440718 6719 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Bootstrap starting.
I20260809 19:20:49.441007 6730 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad: Bootstrap starting.
I20260809 19:20:49.441505 6719 tablet_bootstrap.cc:654] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:49.441795 6730 tablet_bootstrap.cc:654] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:49.441844 6719 log.cc:824] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:49.442145 6730 log.cc:824] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:49.442199 6723 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.442389 6723 raft_consensus.cc:374] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:49.442436 6723 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c075d51984240f6b5d45d0d25a0d595, State: Initialized, Role: FOLLOWER
I20260809 19:20:49.442480 6716 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Bootstrap starting.
I20260809 19:20:49.442574 6723 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.442893 6726 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: No bootstrap required, opened a new log
I20260809 19:20:49.442905 6718 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: No bootstrap required, opened a new log
I20260809 19:20:49.442919 6723 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 19:20:49.442973 6726 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 19:20:49.443030 6001 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
I20260809 19:20:49.443112 6718 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260809 19:20:49.443308 6716 tablet_bootstrap.cc:654] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:49.443662 6716 log.cc:824] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:49.444932 6716 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: No bootstrap required, opened a new log
I20260809 19:20:49.445010 6716 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:20:49.445986 6726 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.446173 6726 raft_consensus.cc:374] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:49.446209 6726 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23183da38752444a9ff5525be50cd404, State: Initialized, Role: FOLLOWER
I20260809 19:20:49.446369 6726 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.446297 6718 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.446465 6718 raft_consensus.cc:374] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:49.446506 6718 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d4177500c7a4f41afce5190ff786506, State: Initialized, Role: FOLLOWER
I20260809 19:20:49.446697 6718 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.446781 6726 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 19:20:49.446897 6678 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
I20260809 19:20:49.447001 6718 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 19:20:49.447151 5558 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
I20260809 19:20:49.447561 6719 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: No bootstrap required, opened a new log
I20260809 19:20:49.447693 6719 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 19:20:49.447769 6716 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.447943 6716 raft_consensus.cc:374] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:49.447988 6716 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 757ec8723e924451a89abb5cc62c1073, State: Initialized, Role: FOLLOWER
I20260809 19:20:49.448132 6716 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.448264 6730 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad: No bootstrap required, opened a new log
I20260809 19:20:49.448340 6730 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260809 19:20:49.448472 6716 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 19:20:49.448609 6236 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
I20260809 19:20:49.450563 6719 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.450577 6730 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.450719 6730 raft_consensus.cc:374] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:49.450722 6719 raft_consensus.cc:374] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:49.450738 6730 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4c4b93755eb4aa48c04795ad08e09ad, State: Initialized, Role: FOLLOWER
I20260809 19:20:49.450781 6719 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd68273a4b314386ae0f5489e2f1d468, State: Initialized, Role: FOLLOWER
I20260809 19:20:49.450884 6730 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.450894 6719 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.451256 6730 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:20:49.451264 6719 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:20:49.451298 6457 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
I20260809 19:20:49.451607 5348 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
W20260809 19:20:49.460688 5563 tablet.cc:2369] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:20:49.465745 6002 tablet.cc:2369] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:20:49.496417 6745 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:20:49.496595 6745 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.497108 6745 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:49.501948 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:49.502139 6593 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 0.
W20260809 19:20:49.502511 6459 tablet.cc:2369] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:20:49.503240 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:20:49.503399 5696 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 0.
I20260809 19:20:49.504026 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:49.504230 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 0.
I20260809 19:20:49.504287 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:49.504446 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 0.
I20260809 19:20:49.504645 5175 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters:
I20260809 19:20:49.504668 5922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:49.504820 5922 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 0.
I20260809 19:20:49.504845 6745 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:20:49.504901 6745 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:20:49.504925 6745 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:49.504997 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:49.505144 6157 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 0.
I20260809 19:20:49.505817 6745 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.506021 6745 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 1 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:49.506203 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073"
I20260809 19:20:49.506273 5922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:20:49.506239 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:20:49.506285 6157 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:49.506347 5922 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:49.506347 6593 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:49.506451 5696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:20:49.506531 5696 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:49.507262 6593 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 1.
I20260809 19:20:49.507262 5922 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 1.
I20260809 19:20:49.507264 6157 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 1.
I20260809 19:20:49.507499 5696 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 1.
I20260809 19:20:49.507579 5177 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, dd68273a4b314386ae0f5489e2f1d468; no voters:
I20260809 19:20:49.507608 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:20:49.507612 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:20:49.507691 6382 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:49.507691 5482 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:49.508460 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 1.
I20260809 19:20:49.508585 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 1.
I20260809 19:20:49.508651 6745 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:20:49.508872 6745 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 1 LEADER]: Becoming Leader. State: Replica: dd68273a4b314386ae0f5489e2f1d468, State: Running, Role: LEADER
I20260809 19:20:49.508957 6745 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:49.510185 5046 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193). New cstate: current_term: 1 leader_uuid: "dd68273a4b314386ae0f5489e2f1d468" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
I20260809 19:20:49.535140 4039 external_mini_cluster.cc:930] 7 TS(s) registered with all masters
I20260809 19:20:49.535746 4039 ts_itest-base.cc:233] Waiting for 1 tablets on tserver dd68273a4b314386ae0f5489e2f1d468 to finish bootstrapping
I20260809 19:20:49.540097 4039 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 7d4177500c7a4f41afce5190ff786506 to finish bootstrapping
I20260809 19:20:49.543735 4039 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 61427f2036ac4f96a9655aef910a7d7a to finish bootstrapping
I20260809 19:20:49.547351 4039 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 6c075d51984240f6b5d45d0d25a0d595 to finish bootstrapping
I20260809 19:20:49.551168 4039 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 757ec8723e924451a89abb5cc62c1073 to finish bootstrapping
I20260809 19:20:49.554375 4039 ts_itest-base.cc:233] Waiting for 1 tablets on tserver d4c4b93755eb4aa48c04795ad08e09ad to finish bootstrapping
I20260809 19:20:49.557511 4039 ts_itest-base.cc:233] Waiting for 1 tablets on tserver 23183da38752444a9ff5525be50cd404 to finish bootstrapping
I20260809 19:20:49.568284 5922 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 1 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:49.568342 6382 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 1 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:49.568527 6157 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 1 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:49.568629 5482 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 1 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:49.569082 6745 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:49.569259 6789 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:49.569339 6593 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 1 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 19:20:49.569355 5696 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 1 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 19:20:49.569657 6755 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:49.570919 6745 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:49.571107 6745 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:49.571242 6745 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:49.575351 6777 mvcc.cc:205] Tried to move back new op lower bound from 7316698110233001984 to 7316698109989515264. Current Snapshot: MvccSnapshot[applied={T|T < 7316698110228652032}]
I20260809 19:20:49.576804 6799 mvcc.cc:205] Tried to move back new op lower bound from 7316698110233001984 to 7316698109989515264. Current Snapshot: MvccSnapshot[applied={T|T < 7316698110228652032}]
I20260809 19:20:49.577877 6792 mvcc.cc:205] Tried to move back new op lower bound from 7316698110233001984 to 7316698109989515264. Current Snapshot: MvccSnapshot[applied={T|T < 7316698110228652032}]
I20260809 19:20:49.580869 6808 mvcc.cc:205] Tried to move back new op lower bound from 7316698110233001984 to 7316698109989515264. Current Snapshot: MvccSnapshot[applied={T|T < 7316698110228652032}]
I20260809 19:20:50.604096 4039 test_util.cc:268] Using random seed: 582393910
I20260809 19:20:50.915452 4039 test_util.cc:268] Using random seed: 582705264
I20260809 19:20:51.320891 4039 test_util.cc:268] Using random seed: 583110704
I20260809 19:20:51.654912 6876 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:20:51.655023 6876 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:51.655426 6876 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:51.657418 6877 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:20:51.657558 6877 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:51.659032 6877 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:51.660048 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 2 candidate_status { last_received { term: 1 index: 862 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:51.660123 6592 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 1.
I20260809 19:20:51.660724 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 2 candidate_status { last_received { term: 1 index: 862 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:51.661489 5695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 2 candidate_status { last_received { term: 1 index: 862 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:20:51.661586 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 2 candidate_status { last_received { term: 1 index: 862 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:51.664853 5695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 2 candidate_status { last_received { term: 1 index: 862 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:20:51.664956 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 2 candidate_status { last_received { term: 1 index: 862 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:51.665030 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 2 candidate_status { last_received { term: 1 index: 862 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:51.665107 6157 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 1.
I20260809 19:20:51.665349 6034 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 757ec8723e924451a89abb5cc62c1073; no voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad
I20260809 19:20:51.665477 6876 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:20:51.665589 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 2 candidate_status { last_received { term: 1 index: 862 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:51.667598 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 2 candidate_status { last_received { term: 1 index: 862 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:51.667791 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 2 candidate_status { last_received { term: 1 index: 862 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:51.668071 6486 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 757ec8723e924451a89abb5cc62c1073; no voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad
I20260809 19:20:51.668195 6877 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:20:51.825996 4039 test_util.cc:268] Using random seed: 583615809
I20260809 19:20:51.997184 6909 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:20:51.997284 6909 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:51.997539 6914 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:20:51.997653 6914 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:51.999406 6909 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:52.000213 6914 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545)
I20260809 19:20:52.000510 6944 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:20:52.000636 6944 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:52.003438 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 2 candidate_status { last_received { term: 1 index: 468 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:52.003578 6592 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for term 2 because replica has last-logged OpId of term: 1 index: 862, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 468.
I20260809 19:20:52.004380 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 2 candidate_status { last_received { term: 1 index: 468 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:52.004499 6157 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for term 2 because replica has last-logged OpId of term: 1 index: 862, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 468.
I20260809 19:20:52.005404 5695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 316 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:20:52.005505 5695 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4c4b93755eb4aa48c04795ad08e09ad for term 2 because replica has last-logged OpId of term: 1 index: 855, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 316.
I20260809 19:20:52.006649 5695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 2 candidate_status { last_received { term: 1 index: 468 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:20:52.006744 5695 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for term 2 because replica has last-logged OpId of term: 1 index: 855, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 468.
I20260809 19:20:52.007306 6944 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:52.007283 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 2 candidate_status { last_received { term: 1 index: 468 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:52.007385 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 1.
I20260809 19:20:52.008222 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 316 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:52.008315 6157 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4c4b93755eb4aa48c04795ad08e09ad for term 2 because replica has last-logged OpId of term: 1 index: 862, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 316.
I20260809 19:20:52.010526 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 316 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:52.010627 5918 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4c4b93755eb4aa48c04795ad08e09ad for term 2 because replica has last-logged OpId of term: 1 index: 468, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 316.
I20260809 19:20:52.010763 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 316 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:52.010879 5482 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4c4b93755eb4aa48c04795ad08e09ad for term 2 because replica has last-logged OpId of term: 1 index: 653, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 316.
I20260809 19:20:52.011070 6267 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: d4c4b93755eb4aa48c04795ad08e09ad; no voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506
I20260809 19:20:52.011253 6914 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:20:52.012025 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 855 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:52.012127 6592 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a for term 2 because replica has last-logged OpId of term: 1 index: 862, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 855.
I20260809 19:20:52.012346 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 2 candidate_status { last_received { term: 1 index: 468 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:52.012463 5482 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for term 2 because replica has last-logged OpId of term: 1 index: 653, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 468.
I20260809 19:20:52.012797 5803 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506
I20260809 19:20:52.012949 6909 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:20:52.012938 6948 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:20:52.013000 6948 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:52.013907 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 316 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:52.014009 6592 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4c4b93755eb4aa48c04795ad08e09ad for term 2 because replica has last-logged OpId of term: 1 index: 862, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 316.
I20260809 19:20:52.014096 6948 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:52.015076 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 855 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:52.015182 6157 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a for term 2 because replica has last-logged OpId of term: 1 index: 862, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 855.
I20260809 19:20:52.016808 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 855 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:52.016884 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 1.
I20260809 19:20:52.017470 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 855 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:52.017560 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 1.
I20260809 19:20:52.019155 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 2 candidate_status { last_received { term: 1 index: 653 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:52.019234 6592 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 for term 2 because replica has last-logged OpId of term: 1 index: 862, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 653.
I20260809 19:20:52.019644 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 2 candidate_status { last_received { term: 1 index: 653 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:52.019747 6157 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 for term 2 because replica has last-logged OpId of term: 1 index: 862, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 653.
I20260809 19:20:52.020622 5695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 2 candidate_status { last_received { term: 1 index: 653 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:20:52.020727 5695 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 for term 2 because replica has last-logged OpId of term: 1 index: 855, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 653.
I20260809 19:20:52.021878 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 2 candidate_status { last_received { term: 1 index: 653 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:52.021992 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 1.
I20260809 19:20:52.022347 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 2 candidate_status { last_received { term: 1 index: 653 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:52.022446 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 1.
I20260809 19:20:52.022610 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 855 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:52.022672 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 1.
I20260809 19:20:52.022943 5587 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 757ec8723e924451a89abb5cc62c1073
I20260809 19:20:52.023089 6944 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 19:20:52.023133 6944 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 1 FOLLOWER]: Starting leader election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:20:52.023162 6944 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:20:52.023859 6944 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:52.024050 6944 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 2 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:52.024205 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 855 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073"
I20260809 19:20:52.024238 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 855 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:20:52.024296 6157 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:20:52.024320 6592 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:20:52.024292 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 855 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:20:52.024361 5482 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:20:52.025039 6157 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a for term 2 because replica has last-logged OpId of term: 1 index: 862, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 855.
I20260809 19:20:52.025163 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 2.
I20260809 19:20:52.025189 6592 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a for term 2 because replica has last-logged OpId of term: 1 index: 862, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 855.
I20260809 19:20:52.025168 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 855 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:20:52.025242 6382 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:20:52.025436 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 855 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:20:52.025498 5918 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:20:52.026244 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 2.
I20260809 19:20:52.026348 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 2.
I20260809 19:20:52.026479 5587 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 757ec8723e924451a89abb5cc62c1073
I20260809 19:20:52.026607 6944 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:20:52.026846 6944 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 2 LEADER]: Becoming Leader. State: Replica: 61427f2036ac4f96a9655aef910a7d7a, State: Running, Role: LEADER
I20260809 19:20:52.026986 6944 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 851, Committed index: 851, Last appended: 1.855, Last appended by leader: 855, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:52.027825 5042 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a reported cstate change: term changed from 1 to 2, leader changed from dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193) to 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195). New cstate: current_term: 2 leader_uuid: "61427f2036ac4f96a9655aef910a7d7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
W20260809 19:20:52.331598 5175 connection.cc:232] RPC call timeout handler was delayed by -1.6108104900922626s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260809 19:20:52.331722 5175 connection.cc:232] RPC call timeout handler was delayed by -1.3902498160023242s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260809 19:20:52.331763 5175 connection.cc:232] RPC call timeout handler was delayed by -1.2801488081458956s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260809 19:20:52.331899 5177 connection.cc:232] RPC call timeout handler was delayed by -1.5013166069984436s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260809 19:20:52.333092 6157 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 2 FOLLOWER]: Rejecting Update request from peer dd68273a4b314386ae0f5489e2f1d468 for earlier term 1. Current term is 2. Ops: []
I20260809 19:20:52.334231 5695 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 2 LEADER]: Rejecting Update request from peer dd68273a4b314386ae0f5489e2f1d468 for earlier term 1. Current term is 2. Ops: [1.856-1.862]
I20260809 19:20:52.334597 6787 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: INVALID_TERM, Last received: 1.855, Next index: 856, Last known committed idx: 851, Time since last communication: 0.000s
I20260809 19:20:52.334707 6787 consensus_queue.cc:1225] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Peer 61427f2036ac4f96a9655aef910a7d7a log is divergent from this leader: its last log entry 2.856 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 851
I20260809 19:20:52.336449 6592 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 2 FOLLOWER]: Rejecting Update request from peer dd68273a4b314386ae0f5489e2f1d468 for earlier term 1. Current term is 2. Ops: []
I20260809 19:20:52.336695 6755 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: INVALID_TERM, Last received: 1.862, Next index: 863, Last known committed idx: 854, Time since last communication: 0.000s
I20260809 19:20:52.337021 6784 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: INVALID_TERM, Last received: 1.862, Next index: 863, Last known committed idx: 854, Time since last communication: 0.000s
I20260809 19:20:52.337844 5482 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 2 FOLLOWER]: Rejecting Update request from peer dd68273a4b314386ae0f5489e2f1d468 for earlier term 1. Current term is 2. Ops: [1.654-1.862]
I20260809 19:20:52.337998 6788 raft_consensus.cc:3032] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 19:20:52.338055 6788 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd68273a4b314386ae0f5489e2f1d468, State: Running, Role: LEADER
I20260809 19:20:52.338111 6788 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 855, Committed index: 855, Last appended: 1.862, Last appended by leader: 862, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:52.338270 6788 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:20:52.339406 6157 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 2 FOLLOWER]: Rejecting Update request from peer dd68273a4b314386ae0f5489e2f1d468 for earlier term 1. Current term is 2. Ops: []
I20260809 19:20:52.339952 5695 raft_consensus.cc:1203] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 2 LEADER]: Deduplicated request from leader. Original: 1.851->[1.852-1.862] Dedup: 1.855->[1.856-1.862]
I20260809 19:20:52.340101 5695 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 2 LEADER]: Rejecting Update request from peer dd68273a4b314386ae0f5489e2f1d468 for earlier term 1. Current term is 2. Ops: [1.852-1.862]
I20260809 19:20:52.341845 5918 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 2 FOLLOWER]: Rejecting Update request from peer dd68273a4b314386ae0f5489e2f1d468 for earlier term 1. Current term is 2. Ops: [1.469-1.862]
I20260809 19:20:52.342483 6382 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 2 FOLLOWER]: Rejecting Update request from peer dd68273a4b314386ae0f5489e2f1d468 for earlier term 1. Current term is 2. Ops: [1.317-1.862]
I20260809 19:20:52.342700 5271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 855 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:20:52.342839 5271 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a for term 2 because replica has last-logged OpId of term: 1 index: 862, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 855.
I20260809 19:20:52.342999 5270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 855 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:20:52.343057 5270 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a for term 2 because replica has last-logged OpId of term: 1 index: 862, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 855.
I20260809 19:20:52.343802 6986 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57484 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=854, attempt_no=0}) took 2192 ms. Trace:
I20260809 19:20:52.343854 5270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 2 candidate_status { last_received { term: 1 index: 468 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:20:52.343945 5270 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for term 2 because replica has last-logged OpId of term: 1 index: 862, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 468.
I20260809 19:20:52.343918 6986 rpcz_store.cc:276] 0809 19:20:50.150852 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:50.150874 (+ 22us) service_pool.cc:227] Handling call
0809 19:20:52.343795 (+2192921us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:50.151130 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:50.151138 (+ 8us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:50.151138 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:20:50.151138 (+ 0us) tablet.cc:660] Decoding operations
0809 19:20:50.151152 (+ 14us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:50.151154 (+ 2us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:50.151155 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:50.151163 (+ 8us) tablet.cc:699] Row locks acquired
0809 19:20:50.151163 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:20:50.151168 (+ 5us) write_op.cc:270] Start()
0809 19:20:50.151170 (+ 2us) write_op.cc:276] Timestamp: P: 1786303250151157 usec, L: 0
0809 19:20:50.151170 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:20:50.151192 (+ 22us) log.cc:842] Serialized 686 byte log entry
0809 19:20:52.337751 (+2186559us) op_driver.cc:444] REPLICATION: finished
0809 19:20:52.343587 (+ 5836us) write_op.cc:301] APPLY: starting
0809 19:20:52.343592 (+ 5us) tablet.cc:1365] starting BulkCheckPresence
0809 19:20:52.343599 (+ 7us) tablet.cc:1368] finished BulkCheckPresence
0809 19:20:52.343610 (+ 11us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:20:52.343642 (+ 32us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:20:52.343647 (+ 5us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:20:52.343648 (+ 1us) write_op.cc:312] APPLY: finished
0809 19:20:52.343669 (+ 21us) log.cc:842] Serialized 77 byte log entry
0809 19:20:52.343769 (+ 100us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:52.343774 (+ 5us) write_op.cc:454] Released schema lock
0809 19:20:52.343775 (+ 1us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":5825,"num_ops":10,"prepare.queue_time_us":193,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":98,"raft.queue_time_us":52,"raft.run_cpu_time_us":46,"raft.run_wall_time_us":47,"replication_time_us":2186577,"thread_start_us":170,"threads_started":2}]]}
W20260809 19:20:52.346200 5213 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:57484: Illegal state: replica dd68273a4b314386ae0f5489e2f1d468 is not leader of this config: current role FOLLOWER
I20260809 19:20:52.346567 5270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 2 candidate_status { last_received { term: 1 index: 316 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:20:52.346643 5270 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4c4b93755eb4aa48c04795ad08e09ad for term 2 because replica has last-logged OpId of term: 1 index: 862, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 316.
I20260809 19:20:52.346830 5270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 2 candidate_status { last_received { term: 1 index: 653 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:20:52.346909 5270 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 for term 2 because replica has last-logged OpId of term: 1 index: 862, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 653.
I20260809 19:20:52.347060 5375 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:20:52.347251 6948 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:20:52.347400 5270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 2 candidate_status { last_received { term: 1 index: 862 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:20:52.347481 5270 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 2.
I20260809 19:20:52.348183 5270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 2 candidate_status { last_received { term: 1 index: 862 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:20:52.348265 5270 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 2.
W20260809 19:20:52.352630 6083 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:20:52.361255 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:20:52.370455 5482 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 2 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 1 index: 653. Preceding OpId from leader: term: 2 index: 857. (index mismatch)
I20260809 19:20:52.370527 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 2 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 1 index: 468. Preceding OpId from leader: term: 2 index: 857. (index mismatch)
I20260809 19:20:52.370572 5270 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 2 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 1 index: 862. Preceding OpId from leader: term: 2 index: 857. (term mismatch)
I20260809 19:20:52.370628 5270 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting all ops after (but not including) 856
I20260809 19:20:52.370658 5270 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 1.857
I20260809 19:20:52.370692 6157 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 2 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 1 index: 862. Preceding OpId from leader: term: 2 index: 857. (term mismatch)
I20260809 19:20:52.370739 6157 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting all ops after (but not including) 856
I20260809 19:20:52.370714 6592 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 2 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 1 index: 862. Preceding OpId from leader: term: 2 index: 857. (term mismatch)
W20260809 19:20:52.370734 5270 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:57484: Aborted: Op aborted by new leader
I20260809 19:20:52.370769 6592 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting all ops after (but not including) 856
I20260809 19:20:52.370787 5270 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57484 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=855, attempt_no=0}) took 2219 ms. Trace:
I20260809 19:20:52.370759 6157 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 1.857
I20260809 19:20:52.370796 6592 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 1.857
I20260809 19:20:52.370855 6157 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 1.858
I20260809 19:20:52.370812 5270 rpcz_store.cc:276] 0809 19:20:50.150952 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:50.150977 (+ 25us) service_pool.cc:227] Handling call
0809 19:20:52.370784 (+2219807us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:50.151292 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:50.151295 (+ 3us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:50.151296 (+ 1us) write_op.cc:435] Acquired schema lock
0809 19:20:50.151296 (+ 0us) tablet.cc:660] Decoding operations
0809 19:20:50.151306 (+ 10us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:50.151307 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:50.151307 (+ 0us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:50.151313 (+ 6us) tablet.cc:699] Row locks acquired
0809 19:20:50.151314 (+ 1us) write_op.cc:260] PREPARE: finished
0809 19:20:50.151317 (+ 3us) write_op.cc:270] Start()
0809 19:20:50.151318 (+ 1us) write_op.cc:276] Timestamp: P: 1786303250151307 usec, L: 0
0809 19:20:50.151318 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:20:50.151331 (+ 13us) log.cc:842] Serialized 686 byte log entry
0809 19:20:52.370714 (+2219383us) op_driver.cc:444] REPLICATION: finished
0809 19:20:52.370718 (+ 4us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 19:20:52.370719 (+ 1us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:52.370729 (+ 10us) write_op.cc:454] Released schema lock
0809 19:20:52.370731 (+ 2us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":277,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"replication_time_us":2219385}]]}
I20260809 19:20:52.370896 6157 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 1.859
I20260809 19:20:52.370899 6592 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 1.858
I20260809 19:20:52.370944 6157 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 1.860
I20260809 19:20:52.370947 5270 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 1.858
I20260809 19:20:52.370949 6592 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 1.859
I20260809 19:20:52.370985 6157 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 1.861
I20260809 19:20:52.370992 6592 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 1.860
W20260809 19:20:52.370992 5270 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:57484: Aborted: Op aborted by new leader
I20260809 19:20:52.371021 6157 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 1.862
I20260809 19:20:52.371030 6592 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 1.861
I20260809 19:20:52.371032 5270 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57484 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=856, attempt_no=0}) took 2219 ms. Trace:
I20260809 19:20:52.370981 6995 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 856, Last known committed idx: 651, Time since last communication: 0.000s
I20260809 19:20:52.371085 6592 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 1.862
I20260809 19:20:52.371057 5270 rpcz_store.cc:276] 0809 19:20:50.151194 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:50.151217 (+ 23us) service_pool.cc:227] Handling call
0809 19:20:52.371030 (+2219813us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:50.151341 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:50.151345 (+ 4us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:50.151345 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:20:50.151345 (+ 0us) tablet.cc:660] Decoding operations
0809 19:20:50.151352 (+ 7us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:50.151353 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:50.151353 (+ 0us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:50.151360 (+ 7us) tablet.cc:699] Row locks acquired
0809 19:20:50.151361 (+ 1us) write_op.cc:260] PREPARE: finished
0809 19:20:50.151364 (+ 3us) write_op.cc:270] Start()
0809 19:20:50.151365 (+ 1us) write_op.cc:276] Timestamp: P: 1786303250151354 usec, L: 0
0809 19:20:50.151365 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:20:50.151376 (+ 11us) log.cc:842] Serialized 696 byte log entry
0809 19:20:52.370982 (+2219606us) op_driver.cc:444] REPLICATION: finished
0809 19:20:52.370984 (+ 2us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 19:20:52.370984 (+ 0us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:52.370989 (+ 5us) write_op.cc:454] Released schema lock
0809 19:20:52.370991 (+ 2us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":69,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"replication_time_us":2219612}]]}
I20260809 19:20:52.371167 5270 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 1.859
W20260809 19:20:52.371207 5270 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:57484: Aborted: Op aborted by new leader
I20260809 19:20:52.371243 5270 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57484 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=857, attempt_no=0}) took 2219 ms. Trace:
I20260809 19:20:52.371264 5270 rpcz_store.cc:276] 0809 19:20:50.151290 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:50.151313 (+ 23us) service_pool.cc:227] Handling call
0809 19:20:52.371241 (+2219928us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:50.151386 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:50.151390 (+ 4us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:50.151390 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:20:50.151391 (+ 1us) tablet.cc:660] Decoding operations
0809 19:20:50.151400 (+ 9us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:50.151400 (+ 0us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:50.151401 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:50.151407 (+ 6us) tablet.cc:699] Row locks acquired
0809 19:20:50.151407 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:20:50.151410 (+ 3us) write_op.cc:270] Start()
0809 19:20:50.151411 (+ 1us) write_op.cc:276] Timestamp: P: 1786303250151400 usec, L: 0
0809 19:20:50.151411 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:20:50.151420 (+ 9us) log.cc:842] Serialized 696 byte log entry
0809 19:20:52.371197 (+2219777us) op_driver.cc:444] REPLICATION: finished
0809 19:20:52.371199 (+ 2us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 19:20:52.371200 (+ 1us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:52.371205 (+ 5us) write_op.cc:454] Released schema lock
0809 19:20:52.371206 (+ 1us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":44,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"replication_time_us":2219783}]]}
I20260809 19:20:52.371373 5270 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 1.860
I20260809 19:20:52.371351 6967 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 856, Last known committed idx: 467, Time since last communication: 0.000s
W20260809 19:20:52.371405 5270 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:57484: Aborted: Op aborted by new leader
I20260809 19:20:52.371456 5270 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57484 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=858, attempt_no=0}) took 2218 ms. Trace:
I20260809 19:20:52.371476 5270 rpcz_store.cc:276] 0809 19:20:50.153318 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:50.153344 (+ 26us) service_pool.cc:227] Handling call
0809 19:20:52.371454 (+2218110us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:50.153567 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:50.153574 (+ 7us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:50.153574 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:20:50.153574 (+ 0us) tablet.cc:660] Decoding operations
0809 19:20:50.153585 (+ 11us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:50.153586 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:50.153587 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:50.153597 (+ 10us) tablet.cc:699] Row locks acquired
0809 19:20:50.153598 (+ 1us) write_op.cc:260] PREPARE: finished
0809 19:20:50.153601 (+ 3us) write_op.cc:270] Start()
0809 19:20:50.153603 (+ 2us) write_op.cc:276] Timestamp: P: 1786303250153591 usec, L: 0
0809 19:20:50.153604 (+ 1us) op_driver.cc:331] REPLICATION: starting
0809 19:20:50.153626 (+ 22us) log.cc:842] Serialized 696 byte log entry
0809 19:20:52.371398 (+2217772us) op_driver.cc:444] REPLICATION: finished
0809 19:20:52.371400 (+ 2us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 19:20:52.371401 (+ 1us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:52.371404 (+ 3us) write_op.cc:454] Released schema lock
0809 19:20:52.371405 (+ 1us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":145,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":81,"replication_time_us":2217792}]]}
I20260809 19:20:52.371560 5270 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 1.861
W20260809 19:20:52.371587 5270 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:57484: Aborted: Op aborted by new leader
I20260809 19:20:52.371613 5270 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57484 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=859, attempt_no=0}) took 2218 ms. Trace:
I20260809 19:20:52.371630 5270 rpcz_store.cc:276] 0809 19:20:50.153525 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:50.153548 (+ 23us) service_pool.cc:227] Handling call
0809 19:20:52.371612 (+2218064us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:50.153651 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:50.153655 (+ 4us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:50.153655 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:20:50.153656 (+ 1us) tablet.cc:660] Decoding operations
0809 19:20:50.153663 (+ 7us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:50.153664 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:50.153664 (+ 0us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:50.153674 (+ 10us) tablet.cc:699] Row locks acquired
0809 19:20:50.153674 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:20:50.153677 (+ 3us) write_op.cc:270] Start()
0809 19:20:50.153679 (+ 2us) write_op.cc:276] Timestamp: P: 1786303250153666 usec, L: 0
0809 19:20:50.153679 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:20:50.153715 (+ 36us) log.cc:842] Serialized 696 byte log entry
0809 19:20:52.371581 (+2217866us) op_driver.cc:444] REPLICATION: finished
0809 19:20:52.371583 (+ 2us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 19:20:52.371583 (+ 0us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:52.371587 (+ 4us) write_op.cc:454] Released schema lock
0809 19:20:52.371587 (+ 0us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":61,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":69,"replication_time_us":2217899,"spinlock_wait_cycles":9472}]]}
I20260809 19:20:52.371699 5270 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 1.862
W20260809 19:20:52.371722 5270 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:57484: Aborted: Op aborted by new leader
I20260809 19:20:52.371747 5270 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57484 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=860, attempt_no=0}) took 2217 ms. Trace:
I20260809 19:20:52.371764 5270 rpcz_store.cc:276] 0809 19:20:50.153788 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:50.153812 (+ 24us) service_pool.cc:227] Handling call
0809 19:20:52.371745 (+2217933us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:50.153879 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:50.153885 (+ 6us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:50.153885 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:20:50.153885 (+ 0us) tablet.cc:660] Decoding operations
0809 19:20:50.153894 (+ 9us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:50.153895 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:50.153896 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:50.153905 (+ 9us) tablet.cc:699] Row locks acquired
0809 19:20:50.153905 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:20:50.153908 (+ 3us) write_op.cc:270] Start()
0809 19:20:50.153910 (+ 2us) write_op.cc:276] Timestamp: P: 1786303250153898 usec, L: 0
0809 19:20:50.153910 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:20:50.153930 (+ 20us) log.cc:842] Serialized 696 byte log entry
0809 19:20:52.371717 (+2217787us) op_driver.cc:444] REPLICATION: finished
0809 19:20:52.371719 (+ 2us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 19:20:52.371719 (+ 0us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:52.371722 (+ 3us) write_op.cc:454] Released schema lock
0809 19:20:52.371722 (+ 0us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":17,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":69,"replication_time_us":2217805}]]}
I20260809 19:20:52.372406 6995 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 856, Last known committed idx: 854, Time since last communication: 0.000s
I20260809 19:20:52.372553 6995 consensus_queue.cc:1225] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Peer 23183da38752444a9ff5525be50cd404 log is divergent from this leader: its last log entry 1.856 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 854
I20260809 19:20:52.372648 6944 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 856, Last known committed idx: 855, Time since last communication: 0.000s
I20260809 19:20:52.372716 6944 consensus_queue.cc:1225] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Peer dd68273a4b314386ae0f5489e2f1d468 log is divergent from this leader: its last log entry 1.856 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 855
I20260809 19:20:52.372803 6993 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 856, Last known committed idx: 854, Time since last communication: 0.000s
I20260809 19:20:52.372857 6993 consensus_queue.cc:1225] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Peer 757ec8723e924451a89abb5cc62c1073 log is divergent from this leader: its last log entry 1.856 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 854
I20260809 19:20:52.373139 6157 raft_consensus.cc:1203] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.854->[1.855-2.857] Dedup: 1.855->[2.856-2.857]
I20260809 19:20:52.373222 6157 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting all ops after (but not including) 855
I20260809 19:20:52.373247 6157 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 1.856
I20260809 19:20:52.373275 6592 raft_consensus.cc:1203] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.854->[1.855-2.857] Dedup: 1.855->[2.856-2.857]
I20260809 19:20:52.373323 6592 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting all ops after (but not including) 855
I20260809 19:20:52.373349 6592 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 1.856
I20260809 19:20:52.375115 5270 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting all ops after (but not including) 855
I20260809 19:20:52.375168 5270 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 1.856
W20260809 19:20:52.375226 5270 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:57484: Aborted: Op aborted by new leader
I20260809 19:20:52.375280 5270 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57484 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=853, attempt_no=0}) took 2224 ms. Trace:
I20260809 19:20:52.375306 5270 rpcz_store.cc:276] 0809 19:20:50.150834 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:50.150887 (+ 53us) service_pool.cc:227] Handling call
0809 19:20:52.375277 (+2224390us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:50.151230 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:50.151235 (+ 5us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:50.151236 (+ 1us) write_op.cc:435] Acquired schema lock
0809 19:20:50.151236 (+ 0us) tablet.cc:660] Decoding operations
0809 19:20:50.151244 (+ 8us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:50.151245 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:50.151245 (+ 0us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:50.151255 (+ 10us) tablet.cc:699] Row locks acquired
0809 19:20:50.151255 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:20:50.151259 (+ 4us) write_op.cc:270] Start()
0809 19:20:50.151260 (+ 1us) write_op.cc:276] Timestamp: P: 1786303250151249 usec, L: 0
0809 19:20:50.151261 (+ 1us) op_driver.cc:331] REPLICATION: starting
0809 19:20:50.151281 (+ 20us) log.cc:842] Serialized 696 byte log entry
0809 19:20:52.375200 (+2223919us) op_driver.cc:444] REPLICATION: finished
0809 19:20:52.375202 (+ 2us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 19:20:52.375203 (+ 1us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:52.375222 (+ 19us) write_op.cc:454] Released schema lock
0809 19:20:52.375223 (+ 1us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":288,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":57,"replication_time_us":2223933,"spinlock_wait_cycles":5504}]]}
I20260809 19:20:52.381918 6998 mvcc.cc:205] Tried to move back new op lower bound from 7316698121734860800 to 7316698120303267840. Current Snapshot: MvccSnapshot[applied={T|T < 7316698112619139072}]
I20260809 19:20:52.402668 7011 mvcc.cc:205] Tried to move back new op lower bound from 7316698121706848256 to 7316698120303267840. Current Snapshot: MvccSnapshot[applied={T|T < 7316698121706848256 or (T in {7316698121706848256})}]
I20260809 19:20:53.020699 4039 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKf1XAf/build/release/bin/kudu with pid 5565
W20260809 19:20:53.036259 6681 meta_cache.cc:302] tablet 7c092d80045840deac3151a94e01de0b: replica 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997) has failed: Network error: recv got EOF from 127.3.241.195:36997 (error 108)
I20260809 19:20:53.038714 6382 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 2 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 1 index: 316. Preceding OpId from leader: term: 2 index: 857. (index mismatch)
W20260809 19:20:53.038883 6273 connection.cc:441] server connection from 127.3.241.195:51465 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.195:51465 (request call id 2) could send its response
W20260809 19:20:53.039943 5806 connection.cc:441] server connection from 127.3.241.195:60543 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.195:60543 (request call id 21) could send its response
W20260809 19:20:53.043044 5375 connection.cc:441] server connection from 127.3.241.195:40445 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.195:40445 (request call id 86) could send its response
W20260809 19:20:53.044809 5177 connection.cc:441] server connection from 127.3.241.195:56189 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.195:56189 (request call id 253) could send its response
I20260809 19:20:53.046658 4039 test_util.cc:268] Using random seed: 584836476
I20260809 19:20:53.150173 4039 test_util.cc:268] Using random seed: 584939988
I20260809 19:20:53.354215 4039 test_util.cc:268] Using random seed: 585144028
I20260809 19:20:53.658113 4039 test_util.cc:268] Using random seed: 585447926
I20260809 19:20:54.061798 4039 test_util.cc:268] Using random seed: 585851605
I20260809 19:20:54.192023 7066 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 61427f2036ac4f96a9655aef910a7d7a)
I20260809 19:20:54.192140 7066 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:54.192410 7066 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:54.192462 7067 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 61427f2036ac4f96a9655aef910a7d7a)
I20260809 19:20:54.192584 7067 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:54.192656 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 3 candidate_status { last_received { term: 2 index: 1370 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:54.192762 5271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 3 candidate_status { last_received { term: 2 index: 1370 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:20:54.192875 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 3 candidate_status { last_received { term: 2 index: 1370 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:54.192965 6157 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 2.
I20260809 19:20:54.192970 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 3 candidate_status { last_received { term: 2 index: 1370 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:54.193058 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 2.
I20260809 19:20:54.193274 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 3 candidate_status { last_received { term: 2 index: 1370 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:54.193318 5271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 3 candidate_status { last_received { term: 2 index: 1370 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
W20260809 19:20:54.193388 6484 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:20:54.193655 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 3 candidate_status { last_received { term: 2 index: 1370 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:54.193738 6593 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 2.
I20260809 19:20:54.193701 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 3 candidate_status { last_received { term: 2 index: 1370 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
W20260809 19:20:54.193784 6034 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:20:54.193734 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 3 candidate_status { last_received { term: 2 index: 1370 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:54.194065 7067 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
W20260809 19:20:54.194062 6484 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997): Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:20:54.194118 6484 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:20:54.194294 7066 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 19:20:54.194306 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 3 candidate_status { last_received { term: 2 index: 1370 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
W20260809 19:20:54.194351 6034 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997): Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:20:54.194388 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 2.
I20260809 19:20:54.194396 6034 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 757ec8723e924451a89abb5cc62c1073; no voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:20:54.194479 7067 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 19:20:54.538627 7080 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 61427f2036ac4f96a9655aef910a7d7a)
I20260809 19:20:54.538739 7080 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:54.539022 7080 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:54.539248 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 3 candidate_status { last_received { term: 2 index: 972 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:54.539275 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 3 candidate_status { last_received { term: 2 index: 972 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:54.539304 5271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 3 candidate_status { last_received { term: 2 index: 972 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:20:54.539388 6157 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for term 3 because replica has last-logged OpId of term: 2 index: 1370, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 972.
I20260809 19:20:54.539388 6593 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for term 3 because replica has last-logged OpId of term: 2 index: 1370, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 972.
I20260809 19:20:54.539444 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 3 candidate_status { last_received { term: 2 index: 972 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:54.539541 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 2.
W20260809 19:20:54.539747 5803 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:20:54.539741 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 3 candidate_status { last_received { term: 2 index: 972 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:54.539997 5803 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:20:54.540108 7080 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260809 19:20:54.540334 5803 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997): Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:20:54.542225 7083 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 61427f2036ac4f96a9655aef910a7d7a)
I20260809 19:20:54.542284 7083 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:54.542533 7083 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:54.542704 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 3 candidate_status { last_received { term: 2 index: 1144 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:54.542793 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 2.
I20260809 19:20:54.543028 5271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 3 candidate_status { last_received { term: 2 index: 1144 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
W20260809 19:20:54.543082 5375 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:20:54.543190 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 3 candidate_status { last_received { term: 2 index: 1144 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:54.543282 6157 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 for term 3 because replica has last-logged OpId of term: 2 index: 1370, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1144.
I20260809 19:20:54.543437 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 3 candidate_status { last_received { term: 2 index: 1144 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:54.543445 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 3 candidate_status { last_received { term: 2 index: 1144 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:54.543524 6593 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 for term 3 because replica has last-logged OpId of term: 2 index: 1370, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1144.
I20260809 19:20:54.543516 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 2.
W20260809 19:20:54.543701 5375 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997): Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:20:54.543757 5375 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:20:54.543862 7083 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 19:20:54.544039 7091 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 61427f2036ac4f96a9655aef910a7d7a)
I20260809 19:20:54.544121 7091 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:54.544353 7091 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:54.544464 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 3 candidate_status { last_received { term: 2 index: 1363 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:54.544559 6157 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 for term 3 because replica has last-logged OpId of term: 2 index: 1370, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1363.
I20260809 19:20:54.544545 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 3 candidate_status { last_received { term: 2 index: 1363 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:54.544636 6593 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 for term 3 because replica has last-logged OpId of term: 2 index: 1370, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1363.
I20260809 19:20:54.544704 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 3 candidate_status { last_received { term: 2 index: 1363 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:54.544723 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 3 candidate_status { last_received { term: 2 index: 1363 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:54.544710 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 3 candidate_status { last_received { term: 2 index: 1363 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:54.544780 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 2.
I20260809 19:20:54.544795 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 2.
I20260809 19:20:54.544786 5479 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 2.
I20260809 19:20:54.544998 5175 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters: 23183da38752444a9ff5525be50cd404, 757ec8723e924451a89abb5cc62c1073
I20260809 19:20:54.545113 7091 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 19:20:54.545153 7091 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 61427f2036ac4f96a9655aef910a7d7a)
W20260809 19:20:54.545162 5175 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:20:54.545181 7091 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 2 FOLLOWER]: Advancing to term 3
W20260809 19:20:54.545552 5175 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997): Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:20:54.546063 7091 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:54.546247 7091 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 3 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:54.546378 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 3 candidate_status { last_received { term: 2 index: 1363 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:20:54.546377 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 3 candidate_status { last_received { term: 2 index: 1363 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:20:54.546464 5918 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 2 FOLLOWER]: Advancing to term 3
I20260809 19:20:54.546489 6593 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 2 FOLLOWER]: Advancing to term 3
W20260809 19:20:54.546692 5175 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997): Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:20:54.547063 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 3.
I20260809 19:20:54.547175 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 3 candidate_status { last_received { term: 2 index: 1363 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073"
I20260809 19:20:54.547235 6157 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 2 FOLLOWER]: Advancing to term 3
I20260809 19:20:54.547284 6593 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 for term 3 because replica has last-logged OpId of term: 2 index: 1370, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1363.
I20260809 19:20:54.547451 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 3 candidate_status { last_received { term: 2 index: 1363 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:20:54.547513 5479 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 2 FOLLOWER]: Advancing to term 3
I20260809 19:20:54.547914 6157 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 for term 3 because replica has last-logged OpId of term: 2 index: 1370, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1363.
I20260809 19:20:54.548004 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 3 candidate_status { last_received { term: 2 index: 1363 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:20:54.548084 6382 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 2 FOLLOWER]: Advancing to term 3
I20260809 19:20:54.548229 5479 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 3.
I20260809 19:20:54.548663 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 3.
I20260809 19:20:54.548830 5175 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073
I20260809 19:20:54.548936 7091 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 19:20:54.549065 7091 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 3 LEADER]: Becoming Leader. State: Replica: dd68273a4b314386ae0f5489e2f1d468, State: Running, Role: LEADER
I20260809 19:20:54.549144 7091 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1359, Committed index: 1359, Last appended: 2.1363, Last appended by leader: 1363, Current term: 3, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:54.549892 5043 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 reported cstate change: term changed from 2 to 3, leader changed from 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195) to dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193). New cstate: current_term: 3 leader_uuid: "dd68273a4b314386ae0f5489e2f1d468" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
I20260809 19:20:54.565708 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.195:36997
--local_ip_for_outbound_sockets=127.3.241.195
--tserver_master_addrs=127.3.241.254:44125
--webserver_port=42257
--webserver_interface=127.3.241.195
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260809 19:20:54.652565 7105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:54.652776 7105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:54.652796 7105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:54.653961 7105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:54.654012 7105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.195
I20260809 19:20:54.655344 7105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.195:36997
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.3.241.195
--webserver_port=42257
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.7105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.195
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:54.655583 7105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:54.655887 7105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:54.661463 7112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:54.661521 7110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:54.661499 7105 server_base.cc:1034] running on GCE node
W20260809 19:20:54.661628 7119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:54.661819 7105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:54.662089 7105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:54.663247 7105 hybrid_clock.cc:648] HybridClock initialized: now 1786303254663238 us; error 43 us; skew 500 ppm
I20260809 19:20:54.669420 7105 webserver.cc:460] Webserver started at http://127.3.241.195:42257/ using document root <none> and password file <none>
I20260809 19:20:54.669661 7105 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:54.669749 7105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:54.671003 7105 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:54.671687 7130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:54.671864 7105 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:54.671963 7105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "61427f2036ac4f96a9655aef910a7d7a"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:54.672416 7105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:54.680568 7105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:54.680845 7105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:54.680989 7105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:54.681206 7105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:54.681615 7138 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:20:54.682861 7105 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:20:54.682924 7105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260809 19:20:54.682956 7105 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:20:54.683588 7105 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:20:54.683640 7105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 19:20:54.683704 7138 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Bootstrap starting.
I20260809 19:20:54.691018 7105 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.195:36997
I20260809 19:20:54.691089 7322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.195:36997 every 8 connection(s)
I20260809 19:20:54.691421 7105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260809 19:20:54.692515 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 7105
I20260809 19:20:54.699010 7323 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:20:54.699126 7323 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:54.699327 7323 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:20:54.699985 5043 ts_manager.cc:194] Re-registered known tserver with Master: 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997)
I20260809 19:20:54.700423 5043 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.195:35965
I20260809 19:20:54.710196 7138 log.cc:824] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:54.799541 7138 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1370 overwritten=0 applied=1362 ignored=0} inserts{seen=13600 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260809 19:20:54.799971 7138 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Bootstrap complete.
I20260809 19:20:54.801153 7138 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Time spent bootstrapping tablet: real 0.118s user 0.087s sys 0.028s
I20260809 19:20:54.802716 7138 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 2 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:54.803617 7138 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61427f2036ac4f96a9655aef910a7d7a, State: Initialized, Role: FOLLOWER
I20260809 19:20:54.803833 7138 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1362, Last appended: 2.1370, Last appended by leader: 1370, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:54.804214 7323 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
I20260809 19:20:54.804193 7138 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260809 19:20:54.930691 7252 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 2 FOLLOWER]: Advancing to term 3
I20260809 19:20:54.932075 7252 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 3 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 2 index: 1370. Preceding OpId from leader: term: 3 index: 1364. (term mismatch)
I20260809 19:20:54.932124 7252 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Aborting all ops after (but not including) 1363
I20260809 19:20:54.932147 7252 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Aborting uncommitted WRITE_OP operation due to leader change: 2.1364
I20260809 19:20:54.932288 7252 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Aborting uncommitted WRITE_OP operation due to leader change: 2.1365
I20260809 19:20:54.932365 7252 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Aborting uncommitted WRITE_OP operation due to leader change: 2.1366
I20260809 19:20:54.932426 7252 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Aborting uncommitted WRITE_OP operation due to leader change: 2.1367
I20260809 19:20:54.932476 7252 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Aborting uncommitted WRITE_OP operation due to leader change: 2.1368
I20260809 19:20:54.932524 7252 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Aborting uncommitted WRITE_OP operation due to leader change: 2.1369
I20260809 19:20:54.932591 7252 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Aborting uncommitted WRITE_OP operation due to leader change: 2.1370
I20260809 19:20:54.932977 7091 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1364, Last known committed idx: 1362, Time since last communication: 0.000s
I20260809 19:20:54.944725 6593 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 3 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 2 index: 1370. Preceding OpId from leader: term: 3 index: 1364. (term mismatch)
I20260809 19:20:54.944782 6593 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting all ops after (but not including) 1363
I20260809 19:20:54.944797 6593 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 2.1364
I20260809 19:20:54.944864 6593 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 2.1365
I20260809 19:20:54.944902 6593 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 2.1366
I20260809 19:20:54.944979 6593 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 2.1367
I20260809 19:20:54.945056 6593 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 2.1368
I20260809 19:20:54.945112 6593 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 2.1369
I20260809 19:20:54.945151 6593 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 2.1370
I20260809 19:20:54.945362 7091 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1364, Last known committed idx: 1362, Time since last communication: 0.000s
I20260809 19:20:55.063700 6157 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 3 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 2 index: 1370. Preceding OpId from leader: term: 3 index: 1364. (term mismatch)
I20260809 19:20:55.063828 6157 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting all ops after (but not including) 1363
I20260809 19:20:55.063851 6157 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 2.1364
I20260809 19:20:55.063977 6157 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1361, attempt_no=1}) took 2018 ms. Trace:
I20260809 19:20:55.064034 6157 rpcz_store.cc:276] 0809 19:20:53.045960 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:53.045992 (+ 32us) service_pool.cc:227] Handling call
0809 19:20:55.063972 (+2017980us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260809 19:20:55.064270 6157 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 2.1365
I20260809 19:20:55.064361 6157 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1362, attempt_no=1}) took 2018 ms. Trace:
I20260809 19:20:55.064407 6157 rpcz_store.cc:276] 0809 19:20:53.045864 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:53.045886 (+ 22us) service_pool.cc:227] Handling call
0809 19:20:55.064357 (+2018471us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260809 19:20:55.064520 6157 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 2.1366
I20260809 19:20:55.064585 6157 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1363, attempt_no=1}) took 2018 ms. Trace:
I20260809 19:20:55.064626 6157 rpcz_store.cc:276] 0809 19:20:53.045909 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:53.045931 (+ 22us) service_pool.cc:227] Handling call
0809 19:20:55.064582 (+2018651us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260809 19:20:55.064703 6157 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 2.1367
I20260809 19:20:55.064771 6157 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1364, attempt_no=1}) took 2018 ms. Trace:
I20260809 19:20:55.064806 6157 rpcz_store.cc:276] 0809 19:20:53.046000 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:53.046028 (+ 28us) service_pool.cc:227] Handling call
0809 19:20:55.064768 (+2018740us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260809 19:20:55.064888 6157 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 2.1368
I20260809 19:20:55.064944 6157 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1365, attempt_no=1}) took 2019 ms. Trace:
I20260809 19:20:55.064983 6157 rpcz_store.cc:276] 0809 19:20:53.045826 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:53.045888 (+ 62us) service_pool.cc:227] Handling call
0809 19:20:55.064940 (+2019052us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260809 19:20:55.065060 6157 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 2.1369
I20260809 19:20:55.065115 6157 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1366, attempt_no=1}) took 2019 ms. Trace:
I20260809 19:20:55.065155 6157 rpcz_store.cc:276] 0809 19:20:53.045981 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:53.046002 (+ 21us) service_pool.cc:227] Handling call
0809 19:20:55.065112 (+2019110us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260809 19:20:55.065232 6157 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 2.1370
I20260809 19:20:55.065290 6157 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1367, attempt_no=1}) took 2019 ms. Trace:
I20260809 19:20:55.065330 6157 rpcz_store.cc:276] 0809 19:20:53.046033 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:53.046049 (+ 16us) service_pool.cc:227] Handling call
0809 19:20:55.065287 (+2019238us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260809 19:20:55.065722 7091 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1364, Last known committed idx: 1362, Time since last communication: 0.000s
I20260809 19:20:55.068001 7363 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1360, attempt_no=1}) took 2022 ms. Trace:
W20260809 19:20:55.068050 6081 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.068125 6083 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
I20260809 19:20:55.068077 7363 rpcz_store.cc:276] 0809 19:20:53.045887 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:53.045938 (+ 51us) service_pool.cc:227] Handling call
0809 19:20:55.067992 (+2022054us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 19:20:55.068910 6083 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.069439 6083 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.069556 6080 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.069599 6081 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.071117 6081 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.071748 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.071769 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.071789 6081 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.073417 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.075733 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.075773 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.076913 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.076933 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:20:55.380102 4039 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKf1XAf/build/release/bin/kudu with pid 5148
W20260809 19:20:55.396557 6681 meta_cache.cc:302] tablet 7c092d80045840deac3151a94e01de0b: replica dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757) has failed: Network error: recv got EOF from 127.3.241.193:46757 (error 108)
I20260809 19:20:55.398779 6382 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 3 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 1 index: 316. Preceding OpId from leader: term: 3 index: 1364. (index mismatch)
W20260809 19:20:55.398916 6271 connection.cc:441] server connection from 127.3.241.193:43213 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.193:43213 (request call id 86) could send its response
I20260809 19:20:55.399963 5479 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 3 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 2 index: 1144. Preceding OpId from leader: term: 3 index: 1364. (index mismatch)
W20260809 19:20:55.401345 5377 connection.cc:441] server connection from 127.3.241.193:49189 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.193:49189 (request call id 232) could send its response
I20260809 19:20:55.401648 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 3 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 2 index: 972. Preceding OpId from leader: term: 3 index: 1364. (index mismatch)
W20260809 19:20:55.401647 7133 connection.cc:441] server connection from 127.3.241.193:55033 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.193:55033 (request call id 2) could send its response
W20260809 19:20:55.401770 5804 connection.cc:441] server connection from 127.3.241.193:46241 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.193:46241 (request call id 224) could send its response
I20260809 19:20:55.405268 4039 test_util.cc:268] Using random seed: 587195084
W20260809 19:20:55.407572 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.407573 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.407701 5408 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.407646 5411 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.407878 5407 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.407573 5409 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.408322 5405 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.408349 5404 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.419243 5848 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.419278 5845 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.419314 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.419190 5844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.419448 5850 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.419201 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.419190 5851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.419759 5851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:20:55.430611 6303 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:20:55.430549 6304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:20:55.430538 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:20:55.430536 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:20:55.430552 6310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:20:55.431120 6310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:20:55.431120 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:20:55.442704 7165 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51492: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:20:55.442688 7169 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51492: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:20:55.442688 7167 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51492: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:20:55.442704 7164 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51492: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:20:55.443372 7169 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51492: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:20:55.443408 7167 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51492: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:20:55.448076 7167 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:51492: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
I20260809 19:20:55.509322 4039 test_util.cc:268] Using random seed: 587299136
I20260809 19:20:55.713882 4039 test_util.cc:268] Using random seed: 587503696
I20260809 19:20:56.017868 4039 test_util.cc:268] Using random seed: 587807675
I20260809 19:20:56.264250 7386 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:20:56.264367 7386 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:56.265007 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 4 candidate_status { last_received { term: 2 index: 972 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:56.265043 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 4 candidate_status { last_received { term: 2 index: 972 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:56.265430 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 4 candidate_status { last_received { term: 2 index: 972 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:56.265643 5479 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for term 4 because replica has last-logged OpId of term: 2 index: 1144, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 972.
W20260809 19:20:56.265722 5803 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757): Network error: Client connection negotiation failed: client connection to 127.3.241.193:46757: connect: Connection refused (error 111)
I20260809 19:20:56.265763 7386 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:56.265884 5803 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595; no voters: 23183da38752444a9ff5525be50cd404, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:20:56.265977 7386 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 19:20:56.266000 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 4 candidate_status { last_received { term: 2 index: 972 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:56.266131 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 3.
I20260809 19:20:56.268872 7252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 4 candidate_status { last_received { term: 2 index: 972 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:20:56.421792 4039 test_util.cc:268] Using random seed: 588211599
I20260809 19:20:56.583935 7392 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:20:56.584179 7392 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:56.584533 7392 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:56.584765 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 4 candidate_status { last_received { term: 3 index: 1372 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:56.584776 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 4 candidate_status { last_received { term: 3 index: 1372 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:56.584892 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 3.
I20260809 19:20:56.584913 6593 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 3.
I20260809 19:20:56.585265 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 4 candidate_status { last_received { term: 3 index: 1372 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:56.585378 5479 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 3.
I20260809 19:20:56.585441 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 4 candidate_status { last_received { term: 3 index: 1372 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:56.585593 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 3.
I20260809 19:20:56.585841 6034 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506; no voters:
I20260809 19:20:56.586073 7392 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260809 19:20:56.586146 6034 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757): Network error: Client connection negotiation failed: client connection to 127.3.241.193:46757: connect: Connection refused (error 111)
I20260809 19:20:56.586192 7392 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:20:56.586227 7392 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 3 FOLLOWER]: Advancing to term 4
I20260809 19:20:56.587093 7392 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:56.587503 7392 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 4 election: Requested vote from peers 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:56.587546 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 4 candidate_status { last_received { term: 3 index: 1372 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:20:56.587639 5918 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 3 FOLLOWER]: Advancing to term 4
I20260809 19:20:56.587621 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 4 candidate_status { last_received { term: 3 index: 1372 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:20:56.587693 6593 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 3 FOLLOWER]: Advancing to term 4
I20260809 19:20:56.588378 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 4 candidate_status { last_received { term: 3 index: 1372 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:20:56.588449 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 4.
I20260809 19:20:56.588459 6382 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 3 FOLLOWER]: Advancing to term 4
I20260809 19:20:56.588456 6593 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 4.
I20260809 19:20:56.588562 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 4 candidate_status { last_received { term: 3 index: 1372 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:20:56.588642 5479 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 3 FOLLOWER]: Advancing to term 4
W20260809 19:20:56.588848 6034 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757): Network error: Client connection negotiation failed: client connection to 127.3.241.193:46757: connect: Connection refused (error 111)
I20260809 19:20:56.589310 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 4.
I20260809 19:20:56.589329 7252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 4 candidate_status { last_received { term: 3 index: 1372 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:20:56.589500 6034 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad; no voters: dd68273a4b314386ae0f5489e2f1d468
I20260809 19:20:56.589570 5479 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 4.
I20260809 19:20:56.589607 7392 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 19:20:56.589692 7251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 4 candidate_status { last_received { term: 3 index: 1372 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:20:56.589777 7392 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 4 LEADER]: Becoming Leader. State: Replica: 757ec8723e924451a89abb5cc62c1073, State: Running, Role: LEADER
I20260809 19:20:56.589896 7392 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1364, Committed index: 1364, Last appended: 3.1372, Last appended by leader: 1372, Current term: 4, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:56.590782 5043 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 reported cstate change: term changed from 3 to 4, leader changed from dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193) to 757ec8723e924451a89abb5cc62c1073 (127.3.241.197). New cstate: current_term: 4 leader_uuid: "757ec8723e924451a89abb5cc62c1073" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
I20260809 19:20:56.901232 7407 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:20:56.901399 7407 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:56.901849 7407 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
W20260809 19:20:56.905589 7132 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.193:46757: connect: Connection refused (error 111)
I20260809 19:20:56.907294 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 4 candidate_status { last_received { term: 3 index: 1364 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
W20260809 19:20:56.908434 7132 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757): Network error: Client connection negotiation failed: client connection to 127.3.241.193:46757: connect: Connection refused (error 111)
I20260809 19:20:56.908382 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 4 candidate_status { last_received { term: 3 index: 1364 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:56.908517 5479 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a in current term 4: Already voted for candidate 757ec8723e924451a89abb5cc62c1073 in this term.
I20260809 19:20:56.910003 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 4 candidate_status { last_received { term: 3 index: 1364 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:56.910143 5918 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a in current term 4: Already voted for candidate 757ec8723e924451a89abb5cc62c1073 in this term.
I20260809 19:20:56.910311 7133 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a; no voters: 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:20:56.910538 7407 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 3 FOLLOWER]: Advancing to term 4
I20260809 19:20:56.910616 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 4 candidate_status { last_received { term: 3 index: 1364 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:56.910724 6593 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a in current term 4: Already voted for candidate 757ec8723e924451a89abb5cc62c1073 in this term.
I20260809 19:20:56.911624 7407 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 19:20:56.915493 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 4 candidate_status { last_received { term: 3 index: 1364 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:56.915598 6382 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a in current term 4: Already voted for candidate 757ec8723e924451a89abb5cc62c1073 in this term.
I20260809 19:20:56.926815 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.193:46757
--local_ip_for_outbound_sockets=127.3.241.193
--tserver_master_addrs=127.3.241.254:44125
--webserver_port=33375
--webserver_interface=127.3.241.193
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260809 19:20:56.990623 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 4 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 2 index: 972. Preceding OpId from leader: term: 4 index: 1373. (index mismatch)
I20260809 19:20:56.991058 7392 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1373, Last known committed idx: 965, Time since last communication: 0.000s
W20260809 19:20:57.035509 7426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:57.035706 7426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:57.035729 7426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:57.036844 7426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:57.036888 7426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.193
I20260809 19:20:57.038149 7426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.193:46757
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.3.241.193
--webserver_port=33375
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.7426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.193
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:57.038337 7426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:57.038704 7426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:57.043951 7446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:57.043964 7444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:57.043994 7426 server_base.cc:1034] running on GCE node
W20260809 19:20:57.043962 7442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:57.044512 7426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:57.044772 7426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:57.045958 7426 hybrid_clock.cc:648] HybridClock initialized: now 1786303257045939 us; error 41 us; skew 500 ppm
I20260809 19:20:57.047370 7426 webserver.cc:460] Webserver started at http://127.3.241.193:33375/ using document root <none> and password file <none>
I20260809 19:20:57.047642 7426 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:57.047744 7426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:57.049178 7426 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:20:57.049945 7460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:57.050134 7426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:57.050243 7426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "dd68273a4b314386ae0f5489e2f1d468"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:20:57.050580 7426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:57.068903 7426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:57.069252 7426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:57.069411 7426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:57.069662 7426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:57.070142 7469 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:20:57.071678 7426 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:20:57.071748 7426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:20:57.071815 7426 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:20:57.072566 7426 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:20:57.072625 7426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:57.072686 7469 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Bootstrap starting.
I20260809 19:20:57.079334 7426 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.193:46757
I20260809 19:20:57.079412 7646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.193:46757 every 8 connection(s)
I20260809 19:20:57.079742 7426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260809 19:20:57.085574 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 7426
I20260809 19:20:57.089669 6382 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 4 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 1 index: 316. Preceding OpId from leader: term: 4 index: 1373. (index mismatch)
I20260809 19:20:57.089926 7392 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1373, Last known committed idx: 310, Time since last communication: 0.000s
I20260809 19:20:57.091445 7647 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:20:57.091537 7647 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:57.091725 7647 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:20:57.092342 5043 ts_manager.cc:194] Re-registered known tserver with Master: dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757)
I20260809 19:20:57.092909 5043 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.193:53977
I20260809 19:20:57.102520 7469 log.cc:824] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:57.117081 6593 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 4 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 3 index: 1372. Preceding OpId from leader: term: 4 index: 1373. (index mismatch)
I20260809 19:20:57.117530 7392 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1373, Last known committed idx: 1364, Time since last communication: 0.000s
W20260809 19:20:57.128636 6034 consensus_peers.cc:487] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 -> Peer dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757): Couldn't send request to peer dd68273a4b314386ae0f5489e2f1d468. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 19:20:57.142002 5479 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 4 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 2 index: 1144. Preceding OpId from leader: term: 4 index: 1373. (index mismatch)
I20260809 19:20:57.142215 7251 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 4 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 3 index: 1364. Preceding OpId from leader: term: 4 index: 1373. (index mismatch)
I20260809 19:20:57.142670 7392 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1373, Last known committed idx: 1138, Time since last communication: 0.000s
I20260809 19:20:57.143251 7392 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1373, Last known committed idx: 1364, Time since last communication: 0.000s
I20260809 19:20:57.154361 7686 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1364, attempt_no=6}) took 1728 ms. Trace:
I20260809 19:20:57.155443 7686 rpcz_store.cc:276] 0809 19:20:55.426231 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:55.426290 (+ 59us) service_pool.cc:227] Handling call
0809 19:20:57.154346 (+1728056us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:20:57.157023 7673 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1362, attempt_no=11}) took 1672 ms. Trace:
I20260809 19:20:57.157094 7673 rpcz_store.cc:276] 0809 19:20:55.484947 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:55.484984 (+ 37us) service_pool.cc:227] Handling call
0809 19:20:57.156998 (+1672014us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:20:57.157280 7673 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1367, attempt_no=11}) took 1675 ms. Trace:
I20260809 19:20:57.157318 7673 rpcz_store.cc:276] 0809 19:20:55.481585 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:55.481625 (+ 40us) service_pool.cc:227] Handling call
0809 19:20:57.157277 (+1675652us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:20:57.157482 7673 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1368, attempt_no=8}) took 1700 ms. Trace:
I20260809 19:20:57.157517 7673 rpcz_store.cc:276] 0809 19:20:55.456952 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:55.457003 (+ 51us) service_pool.cc:227] Handling call
0809 19:20:57.157480 (+1700477us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:20:57.157618 7673 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1361, attempt_no=11}) took 1678 ms. Trace:
I20260809 19:20:57.157667 7673 rpcz_store.cc:276] 0809 19:20:55.479371 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:55.479409 (+ 38us) service_pool.cc:227] Handling call
0809 19:20:57.157616 (+1678207us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:20:57.157799 7673 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1366, attempt_no=11}) took 1673 ms. Trace:
I20260809 19:20:57.157827 7673 rpcz_store.cc:276] 0809 19:20:55.483835 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:55.483871 (+ 36us) service_pool.cc:227] Handling call
0809 19:20:57.157797 (+1673926us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:20:57.157913 7673 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1365, attempt_no=11}) took 1679 ms. Trace:
I20260809 19:20:57.157939 7673 rpcz_store.cc:276] 0809 19:20:55.478283 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:55.478330 (+ 47us) service_pool.cc:227] Handling call
0809 19:20:57.157911 (+1679581us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:20:57.158042 7673 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1363, attempt_no=11}) took 1675 ms. Trace:
I20260809 19:20:57.158067 7673 rpcz_store.cc:276] 0809 19:20:55.482696 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:55.482733 (+ 37us) service_pool.cc:227] Handling call
0809 19:20:57.158040 (+1675307us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:20:57.163527 7661 mvcc.cc:205] Tried to move back new op lower bound from 7316698132820381696 to 7316698120303267840. Current Snapshot: MvccSnapshot[applied={T|T < 7316698122901540864}]
I20260809 19:20:57.277807 7469 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Bootstrap replayed 1/1 log segments. Stats: ops{read=1379 overwritten=7 applied=1364 ignored=0} inserts{seen=13610 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260809 19:20:57.278513 7469 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Bootstrap complete.
I20260809 19:20:57.279950 7469 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Time spent bootstrapping tablet: real 0.207s user 0.145s sys 0.039s
I20260809 19:20:57.282001 7469 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 3 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:57.283262 7469 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd68273a4b314386ae0f5489e2f1d468, State: Initialized, Role: FOLLOWER
I20260809 19:20:57.283490 7469 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1364, Last appended: 3.1372, Last appended by leader: 1372, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:57.283922 7469 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260809 19:20:57.284138 7647 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
I20260809 19:20:57.581630 7570 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 3 FOLLOWER]: Advancing to term 4
I20260809 19:20:57.595815 7713 mvcc.cc:205] Tried to move back new op lower bound from 7316698142442098688 to 7316698138993512448. Current Snapshot: MvccSnapshot[applied={T|T < 7316698142206402560 or (T in {7316698142214770688,7316698142215438336,7316698142215819264,7316698142216318976,7316698142217134080,7316698142226132992,7316698142226837504,7316698142226444288,7316698142227169280,7316698142229426176,7316698142229757952,7316698142230028288,7316698142232059904,7316698142242996224,7316698142243225600,7316698142242607104,7316698142245048320,7316698142243983360,7316698142245523456,7316698142248779776,7316698142249115648})}]
I20260809 19:20:57.992951 4039 test_util.cc:268] Using random seed: 589782764
I20260809 19:20:58.298084 4039 test_util.cc:268] Using random seed: 590087895
I20260809 19:20:58.703850 4039 test_util.cc:268] Using random seed: 590493662
I20260809 19:20:58.709383 7723 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:20:58.709475 7723 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:58.709734 7723 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545)
I20260809 19:20:58.709939 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 5 candidate_status { last_received { term: 4 index: 1399 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:58.709977 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 5 candidate_status { last_received { term: 4 index: 1399 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:58.710300 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 5 candidate_status { last_received { term: 4 index: 1399 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:58.713091 7570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 5 candidate_status { last_received { term: 4 index: 1399 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:20:58.713129 7252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 5 candidate_status { last_received { term: 4 index: 1399 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:20:58.713426 6267 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506
I20260809 19:20:58.713582 7723 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 19:20:59.100679 7733 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:20:59.100687 7732 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:20:59.100800 7732 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:59.100804 7733 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:59.101996 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:59.102691 7733 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:59.102679 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:59.102928 7732 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:59.102991 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:59.103096 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 4.
I20260809 19:20:59.105330 7570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:20:59.105439 7570 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 4.
I20260809 19:20:59.107087 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:59.107177 6592 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 4.
I20260809 19:20:59.107818 7252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:20:59.108805 7252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:20:59.109392 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:59.110575 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:59.110679 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 4.
I20260809 19:20:59.110915 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:59.210131 4039 test_util.cc:268] Using random seed: 590999943
I20260809 19:20:59.387146 7763 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:20:59.387280 7763 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:59.387912 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 5 candidate_status { last_received { term: 4 index: 1535 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:59.388049 6592 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for term 5 because replica has last-logged OpId of term: 4 index: 1749, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 1535.
I20260809 19:20:59.388128 7252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 5 candidate_status { last_received { term: 4 index: 1535 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:20:59.388232 7763 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:59.388345 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 5 candidate_status { last_received { term: 4 index: 1535 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:59.388451 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 4.
I20260809 19:20:59.388561 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 5 candidate_status { last_received { term: 4 index: 1535 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:59.391114 7570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 5 candidate_status { last_received { term: 4 index: 1535 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:20:59.391245 7570 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for term 5 because replica has last-logged OpId of term: 4 index: 1749, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 1535.
I20260809 19:20:59.391465 5803 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:20:59.391569 7763 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 19:20:59.392220 7766 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:20:59.392306 7766 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:59.392781 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 5 candidate_status { last_received { term: 4 index: 1738 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:59.392894 6592 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 for term 5 because replica has last-logged OpId of term: 4 index: 1749, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 1738.
I20260809 19:20:59.392967 7767 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:20:59.392970 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 5 candidate_status { last_received { term: 4 index: 1738 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:59.393042 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 4.
I20260809 19:20:59.393033 7767 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:59.393532 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 5 candidate_status { last_received { term: 4 index: 1738 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:59.393576 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 5 candidate_status { last_received { term: 4 index: 1741 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:20:59.393615 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 4.
I20260809 19:20:59.393656 6592 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a for term 5 because replica has last-logged OpId of term: 4 index: 1749, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 1741.
I20260809 19:20:59.394043 7767 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:59.394142 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 5 candidate_status { last_received { term: 4 index: 1741 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:20:59.394224 5479 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 4.
I20260809 19:20:59.394405 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 5 candidate_status { last_received { term: 4 index: 1741 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:20:59.394469 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 4.
I20260809 19:20:59.394590 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 5 candidate_status { last_received { term: 4 index: 1741 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:20:59.394656 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 4.
I20260809 19:20:59.394816 7132 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404
I20260809 19:20:59.394929 7766 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:59.394893 7767 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 19:20:59.394975 7767 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:20:59.395008 7767 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 4 FOLLOWER]: Advancing to term 5
I20260809 19:20:59.395787 7767 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:59.395949 7767 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 5 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:59.396238 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 5 candidate_status { last_received { term: 4 index: 1741 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:20:59.396237 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 5 candidate_status { last_received { term: 4 index: 1741 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:20:59.396309 5918 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 4 FOLLOWER]: Advancing to term 5
I20260809 19:20:59.396309 6592 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 4 FOLLOWER]: Advancing to term 5
I20260809 19:20:59.396865 7252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 5 candidate_status { last_received { term: 4 index: 1738 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:20:59.396960 7252 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 in current term 5: Already voted for candidate 61427f2036ac4f96a9655aef910a7d7a in this term.
I20260809 19:20:59.397147 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 5.
I20260809 19:20:59.397205 6592 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a for term 5 because replica has last-logged OpId of term: 4 index: 1749, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 1741.
I20260809 19:20:59.397425 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 5 candidate_status { last_received { term: 4 index: 1741 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:20:59.397450 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 5 candidate_status { last_received { term: 4 index: 1741 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:20:59.397495 5479 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 4 FOLLOWER]: Advancing to term 5
I20260809 19:20:59.397516 6382 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 4 FOLLOWER]: Advancing to term 5
I20260809 19:20:59.398149 5479 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 5.
I20260809 19:20:59.398214 6382 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 5.
I20260809 19:20:59.398345 7132 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404
I20260809 19:20:59.398490 7767 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 19:20:59.398603 7767 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 5 LEADER]: Becoming Leader. State: Replica: 61427f2036ac4f96a9655aef910a7d7a, State: Running, Role: LEADER
I20260809 19:20:59.398626 7570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 5 candidate_status { last_received { term: 4 index: 1738 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:20:59.398712 7570 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 for term 5 because replica has last-logged OpId of term: 4 index: 1749, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 1738.
I20260809 19:20:59.398696 7767 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1741, Committed index: 1741, Last appended: 4.1741, Last appended by leader: 1741, Current term: 5, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:59.399520 5043 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a reported cstate change: term changed from 4 to 5, leader changed from 757ec8723e924451a89abb5cc62c1073 (127.3.241.197) to 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195). New cstate: current_term: 5 leader_uuid: "61427f2036ac4f96a9655aef910a7d7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
I20260809 19:20:59.400492 7570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 5 candidate_status { last_received { term: 4 index: 1741 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:20:59.400511 7569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 5 candidate_status { last_received { term: 4 index: 1741 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:20:59.400597 7570 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a for term 5 because replica has last-logged OpId of term: 4 index: 1749, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 1741.
W20260809 19:20:59.716207 6039 connection.cc:232] RPC call timeout handler was delayed by -1.4957977321464568s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260809 19:20:59.716389 6157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 5 candidate_status { last_received { term: 4 index: 1738 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:59.716833 6153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 5 candidate_status { last_received { term: 4 index: 1399 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
W20260809 19:20:59.716073 6034 connection.cc:232] RPC call timeout handler was delayed by -1.6075168531388044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260809 19:20:59.717638 6034 connection.cc:232] RPC call timeout handler was delayed by -1.3853441930841655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260809 19:20:59.717681 6034 connection.cc:232] RPC call timeout handler was delayed by -1.2208903920836747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260809 19:20:59.718030 6153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 5 candidate_status { last_received { term: 4 index: 1741 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:59.718894 5379 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:20:59.719058 6150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 5 candidate_status { last_received { term: 4 index: 1535 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:59.719177 6139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:59.719244 6146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 5 candidate_status { last_received { term: 4 index: 1741 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073"
I20260809 19:20:59.719249 5918 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 5 FOLLOWER]: Rejecting Update request from peer 757ec8723e924451a89abb5cc62c1073 for earlier term 4. Current term is 5. Ops: [4.1536-4.1749]
I20260809 19:20:59.719249 7766 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 19:20:59.719439 6592 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 5 FOLLOWER]: Rejecting Update request from peer 757ec8723e924451a89abb5cc62c1073 for earlier term 4. Current term is 5. Ops: []
I20260809 19:20:59.719751 7392 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: INVALID_TERM, Last received: 4.1535, Next index: 1536, Last known committed idx: 1534, Time since last communication: 0.000s
I20260809 19:20:59.719856 7392 raft_consensus.cc:3032] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 19:20:59.719887 7392 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 757ec8723e924451a89abb5cc62c1073, State: Running, Role: LEADER
I20260809 19:20:59.719933 7392 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1741, Committed index: 1741, Last appended: 4.1749, Last appended by leader: 1749, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:59.720300 7252 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 5 LEADER]: Rejecting Update request from peer 757ec8723e924451a89abb5cc62c1073 for earlier term 4. Current term is 5. Ops: [4.1742-4.1749]
I20260809 19:20:59.720364 5479 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 5 FOLLOWER]: Rejecting Update request from peer 757ec8723e924451a89abb5cc62c1073 for earlier term 4. Current term is 5. Ops: [4.1739-4.1749]
I20260809 19:20:59.720126 7392 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 4 FOLLOWER]: Advancing to term 5
I20260809 19:20:59.720564 6487 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 23183da38752444a9ff5525be50cd404, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506
I20260809 19:20:59.721325 7789 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 19:20:59.722748 6382 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 5 FOLLOWER]: Rejecting Update request from peer 757ec8723e924451a89abb5cc62c1073 for earlier term 4. Current term is 5. Ops: [4.1400-4.1749]
I20260809 19:20:59.723563 6146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:20:59.723665 6146 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 5.
I20260809 19:20:59.723840 7465 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506
I20260809 19:20:59.724082 7793 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 19:20:59.724200 7793 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:20:59.724241 7793 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 4 FOLLOWER]: Advancing to term 5
I20260809 19:20:59.725138 7793 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:20:59.725371 7793 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 5 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:20:59.725558 6146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073"
I20260809 19:20:59.725580 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:20:59.725647 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:20:59.725731 5479 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 in current term 5: Already voted for candidate 61427f2036ac4f96a9655aef910a7d7a in this term.
I20260809 19:20:59.725710 7252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:20:59.725886 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:20:59.725965 5918 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 in current term 5: Already voted for candidate 61427f2036ac4f96a9655aef910a7d7a in this term.
I20260809 19:20:59.726284 6146 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 5.
I20260809 19:20:59.726392 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 5 candidate_status { last_received { term: 4 index: 1749 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:20:59.726500 6382 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 in current term 5: Already voted for candidate 61427f2036ac4f96a9655aef910a7d7a in this term.
I20260809 19:20:59.726516 6592 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 5.
I20260809 19:20:59.726828 7461 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 757ec8723e924451a89abb5cc62c1073, dd68273a4b314386ae0f5489e2f1d468; no voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad
I20260809 19:20:59.726935 7793 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 19:20:59.814880 6592 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 5 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 4 index: 1749. Preceding OpId from leader: term: 5 index: 1742. (term mismatch)
I20260809 19:20:59.814980 6592 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting all ops after (but not including) 1741
I20260809 19:20:59.814998 6592 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 4.1742
I20260809 19:20:59.815054 6592 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 4.1743
I20260809 19:20:59.815079 6592 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 4.1744
I20260809 19:20:59.815109 6592 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 4.1745
I20260809 19:20:59.815132 6592 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 4.1746
I20260809 19:20:59.815161 6592 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 4.1747
I20260809 19:20:59.815188 6592 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 4.1748
I20260809 19:20:59.815214 6592 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 4.1749
I20260809 19:20:59.815532 7767 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1742, Last known committed idx: 1741, Time since last communication: 0.000s
I20260809 19:20:59.900395 5479 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 5 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 4 index: 1738. Preceding OpId from leader: term: 5 index: 1742. (index mismatch)
I20260809 19:20:59.900723 7767 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1742, Last known committed idx: 1732, Time since last communication: 0.000s
I20260809 19:20:59.951735 6146 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 5 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 4 index: 1749. Preceding OpId from leader: term: 5 index: 1742. (term mismatch)
I20260809 19:20:59.951819 6146 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting all ops after (but not including) 1741
I20260809 19:20:59.951849 6146 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 4.1742
W20260809 19:20:59.951894 6146 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Aborted: Op aborted by new leader
I20260809 19:20:59.951948 6146 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1737, attempt_no=0}) took 2354 ms. Trace:
I20260809 19:20:59.951972 6146 rpcz_store.cc:276] 0809 19:20:57.597109 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:57.597135 (+ 26us) service_pool.cc:227] Handling call
0809 19:20:59.951945 (+2354810us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:57.597307 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:57.597319 (+ 12us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:57.597320 (+ 1us) write_op.cc:435] Acquired schema lock
0809 19:20:57.597320 (+ 0us) tablet.cc:660] Decoding operations
0809 19:20:57.597337 (+ 17us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:57.597339 (+ 2us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:57.597340 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:57.597352 (+ 12us) tablet.cc:699] Row locks acquired
0809 19:20:57.597352 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:20:57.597360 (+ 8us) write_op.cc:270] Start()
0809 19:20:57.597364 (+ 4us) write_op.cc:276] Timestamp: P: 1786303257597332 usec, L: 0
0809 19:20:57.597364 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:20:57.597402 (+ 38us) log.cc:842] Serialized 696 byte log entry
0809 19:20:59.951877 (+2354475us) op_driver.cc:444] REPLICATION: finished
0809 19:20:59.951880 (+ 3us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 19:20:59.951881 (+ 1us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:59.951891 (+ 10us) write_op.cc:454] Released schema lock
0809 19:20:59.951893 (+ 2us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":26,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":124,"raft.queue_time_us":46,"raft.run_cpu_time_us":42,"raft.run_wall_time_us":42,"replication_time_us":2354502}]]}
I20260809 19:20:59.952214 6146 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 4.1743
W20260809 19:20:59.952277 6146 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Aborted: Op aborted by new leader
I20260809 19:20:59.952330 6146 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1738, attempt_no=0}) took 2354 ms. Trace:
I20260809 19:20:59.952355 6146 rpcz_store.cc:276] 0809 19:20:57.597371 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:57.597389 (+ 18us) service_pool.cc:227] Handling call
0809 19:20:59.952328 (+2354939us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:57.597670 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:57.597681 (+ 11us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:57.597682 (+ 1us) write_op.cc:435] Acquired schema lock
0809 19:20:57.597682 (+ 0us) tablet.cc:660] Decoding operations
0809 19:20:57.597707 (+ 25us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:57.597708 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:57.597709 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:57.597720 (+ 11us) tablet.cc:699] Row locks acquired
0809 19:20:57.597720 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:20:57.597725 (+ 5us) write_op.cc:270] Start()
0809 19:20:57.597727 (+ 2us) write_op.cc:276] Timestamp: P: 1786303257597697 usec, L: 0
0809 19:20:57.597727 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:20:57.597751 (+ 24us) log.cc:842] Serialized 696 byte log entry
0809 19:20:59.952268 (+2354517us) op_driver.cc:444] REPLICATION: finished
0809 19:20:59.952269 (+ 1us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 19:20:59.952270 (+ 1us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:59.952275 (+ 5us) write_op.cc:454] Released schema lock
0809 19:20:59.952276 (+ 1us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":219,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":95,"replication_time_us":2354535}]]}
I20260809 19:20:59.952493 6146 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 4.1744
W20260809 19:20:59.952540 6146 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Aborted: Op aborted by new leader
I20260809 19:20:59.952579 6146 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1739, attempt_no=0}) took 2354 ms. Trace:
I20260809 19:20:59.952607 6146 rpcz_store.cc:276] 0809 19:20:57.598245 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:57.598293 (+ 48us) service_pool.cc:227] Handling call
0809 19:20:59.952577 (+2354284us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:57.598383 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:57.598389 (+ 6us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:57.598390 (+ 1us) write_op.cc:435] Acquired schema lock
0809 19:20:57.598390 (+ 0us) tablet.cc:660] Decoding operations
0809 19:20:57.598402 (+ 12us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:57.598403 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:57.598404 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:57.598414 (+ 10us) tablet.cc:699] Row locks acquired
0809 19:20:57.598415 (+ 1us) write_op.cc:260] PREPARE: finished
0809 19:20:57.598418 (+ 3us) write_op.cc:270] Start()
0809 19:20:57.598420 (+ 2us) write_op.cc:276] Timestamp: P: 1786303257598390 usec, L: 0
0809 19:20:57.598420 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:20:57.598453 (+ 33us) log.cc:842] Serialized 696 byte log entry
0809 19:20:59.952529 (+2354076us) op_driver.cc:444] REPLICATION: finished
0809 19:20:59.952531 (+ 2us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 19:20:59.952532 (+ 1us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:59.952538 (+ 6us) write_op.cc:454] Released schema lock
0809 19:20:59.952539 (+ 1us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":17,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":105,"replication_time_us":2354105}]]}
I20260809 19:20:59.952718 6146 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 4.1745
W20260809 19:20:59.952778 6146 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Aborted: Op aborted by new leader
I20260809 19:20:59.952819 6146 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1742, attempt_no=0}) took 2354 ms. Trace:
I20260809 19:20:59.952844 6146 rpcz_store.cc:276] 0809 19:20:57.598317 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:57.598367 (+ 50us) service_pool.cc:227] Handling call
0809 19:20:59.952818 (+2354451us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:57.598490 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:57.598495 (+ 5us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:57.598495 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:20:57.598495 (+ 0us) tablet.cc:660] Decoding operations
0809 19:20:57.598503 (+ 8us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:57.598504 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:57.598504 (+ 0us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:57.598511 (+ 7us) tablet.cc:699] Row locks acquired
0809 19:20:57.598511 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:20:57.598514 (+ 3us) write_op.cc:270] Start()
0809 19:20:57.598516 (+ 2us) write_op.cc:276] Timestamp: P: 1786303257598486 usec, L: 0
0809 19:20:57.598516 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:20:57.598531 (+ 15us) log.cc:842] Serialized 686 byte log entry
0809 19:20:59.952770 (+2354239us) op_driver.cc:444] REPLICATION: finished
0809 19:20:59.952771 (+ 1us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 19:20:59.952772 (+ 1us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:59.952777 (+ 5us) write_op.cc:454] Released schema lock
0809 19:20:59.952777 (+ 0us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":88,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"replication_time_us":2354250}]]}
I20260809 19:20:59.952953 6146 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 4.1746
W20260809 19:20:59.952996 6146 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Aborted: Op aborted by new leader
I20260809 19:20:59.953032 6146 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1741, attempt_no=0}) took 2354 ms. Trace:
I20260809 19:20:59.953058 6146 rpcz_store.cc:276] 0809 19:20:57.598301 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:57.598391 (+ 90us) service_pool.cc:227] Handling call
0809 19:20:59.953030 (+2354639us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:57.598541 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:57.598545 (+ 4us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:57.598545 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:20:57.598545 (+ 0us) tablet.cc:660] Decoding operations
0809 19:20:57.598556 (+ 11us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:57.598556 (+ 0us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:57.598557 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:57.598566 (+ 9us) tablet.cc:699] Row locks acquired
0809 19:20:57.598566 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:20:57.598569 (+ 3us) write_op.cc:270] Start()
0809 19:20:57.598570 (+ 1us) write_op.cc:276] Timestamp: P: 1786303257598541 usec, L: 0
0809 19:20:57.598571 (+ 1us) op_driver.cc:331] REPLICATION: starting
0809 19:20:57.598588 (+ 17us) log.cc:842] Serialized 696 byte log entry
0809 19:20:59.952974 (+2354386us) op_driver.cc:444] REPLICATION: finished
0809 19:20:59.952975 (+ 1us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 19:20:59.952975 (+ 0us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:59.952979 (+ 4us) write_op.cc:454] Released schema lock
0809 19:20:59.952995 (+ 16us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":114,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"replication_time_us":2354400,"spinlock_wait_cycles":11136}]]}
I20260809 19:20:59.953171 6146 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 4.1747
W20260809 19:20:59.953218 6146 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Aborted: Op aborted by new leader
I20260809 19:20:59.953253 6146 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1744, attempt_no=0}) took 2354 ms. Trace:
I20260809 19:20:59.953279 6146 rpcz_store.cc:276] 0809 19:20:57.598347 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:57.598411 (+ 64us) service_pool.cc:227] Handling call
0809 19:20:59.953252 (+2354841us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:57.598596 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:57.598600 (+ 4us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:57.598600 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:20:57.598600 (+ 0us) tablet.cc:660] Decoding operations
0809 19:20:57.598608 (+ 8us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:57.598609 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:57.598610 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:57.598617 (+ 7us) tablet.cc:699] Row locks acquired
0809 19:20:57.598617 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:20:57.598619 (+ 2us) write_op.cc:270] Start()
0809 19:20:57.598620 (+ 1us) write_op.cc:276] Timestamp: P: 1786303257598591 usec, L: 0
0809 19:20:57.598620 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:20:57.598636 (+ 16us) log.cc:842] Serialized 696 byte log entry
0809 19:20:59.953213 (+2354577us) op_driver.cc:444] REPLICATION: finished
0809 19:20:59.953214 (+ 1us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 19:20:59.953214 (+ 0us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:59.953217 (+ 3us) write_op.cc:454] Released schema lock
0809 19:20:59.953218 (+ 1us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":157,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"replication_time_us":2354589}]]}
I20260809 19:20:59.953377 6146 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 4.1748
W20260809 19:20:59.953414 6146 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Aborted: Op aborted by new leader
I20260809 19:20:59.953444 6146 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1743, attempt_no=0}) took 2355 ms. Trace:
I20260809 19:20:59.953473 6146 rpcz_store.cc:276] 0809 19:20:57.598331 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:57.598480 (+ 149us) service_pool.cc:227] Handling call
0809 19:20:59.953443 (+2354963us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:57.598646 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:57.598649 (+ 3us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:57.598649 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:20:57.598650 (+ 1us) tablet.cc:660] Decoding operations
0809 19:20:57.598660 (+ 10us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:57.598661 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:57.598661 (+ 0us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:57.598666 (+ 5us) tablet.cc:699] Row locks acquired
0809 19:20:57.598666 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:20:57.598669 (+ 3us) write_op.cc:270] Start()
0809 19:20:57.598670 (+ 1us) write_op.cc:276] Timestamp: P: 1786303257598641 usec, L: 0
0809 19:20:57.598670 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:20:57.598681 (+ 11us) log.cc:842] Serialized 696 byte log entry
0809 19:20:59.953407 (+2354726us) op_driver.cc:444] REPLICATION: finished
0809 19:20:59.953408 (+ 1us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 19:20:59.953408 (+ 0us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:59.953412 (+ 4us) write_op.cc:454] Released schema lock
0809 19:20:59.953413 (+ 1us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":135,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"replication_time_us":2354733}]]}
I20260809 19:20:59.953586 6146 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 4.1749
W20260809 19:20:59.953629 6146 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Aborted: Op aborted by new leader
I20260809 19:20:59.953668 6146 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1740, attempt_no=0}) took 2355 ms. Trace:
I20260809 19:20:59.953691 6146 rpcz_store.cc:276] 0809 19:20:57.598280 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:20:57.598301 (+ 21us) service_pool.cc:227] Handling call
0809 19:20:59.953667 (+2355366us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:20:57.598734 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:20:57.598738 (+ 4us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:20:57.598738 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:20:57.598739 (+ 1us) tablet.cc:660] Decoding operations
0809 19:20:57.598747 (+ 8us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:20:57.598747 (+ 0us) write_op.cc:641] Partition lock acquired for write op
0809 19:20:57.598748 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:20:57.598755 (+ 7us) tablet.cc:699] Row locks acquired
0809 19:20:57.598756 (+ 1us) write_op.cc:260] PREPARE: finished
0809 19:20:57.598757 (+ 1us) write_op.cc:270] Start()
0809 19:20:57.598759 (+ 2us) write_op.cc:276] Timestamp: P: 1786303257598730 usec, L: 0
0809 19:20:57.598759 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:20:57.598776 (+ 17us) log.cc:842] Serialized 686 byte log entry
0809 19:20:59.953619 (+2354843us) op_driver.cc:444] REPLICATION: finished
0809 19:20:59.953621 (+ 2us) op_driver.cc:372] HandleFailure(Aborted: Op aborted by new leader)
0809 19:20:59.953621 (+ 0us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:20:59.953626 (+ 5us) write_op.cc:454] Released schema lock
0809 19:20:59.953627 (+ 1us) write_op.cc:335] FINISH: Op aborted
Metrics: {"spinlock_wait_cycles":10624,"child_traces":[["op",{"mutex_wait_us":347,"num_ops":10,"prepare.queue_time_us":363,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"replication_time_us":2354857,"spinlock_wait_cycles":3200}]]}
W20260809 19:20:59.953857 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:20:59.954051 7767 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1742, Last known committed idx: 1741, Time since last communication: 0.000s
W20260809 19:20:59.954196 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:20:59.955307 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:20:59.955684 7570 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 5 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 4 index: 1749. Preceding OpId from leader: term: 5 index: 1742. (term mismatch)
I20260809 19:20:59.955756 7570 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting all ops after (but not including) 1741
I20260809 19:20:59.955780 7570 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 4.1742
I20260809 19:20:59.955963 7570 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 4.1743
I20260809 19:20:59.956028 7570 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 4.1744
I20260809 19:20:59.956067 7570 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 4.1745
I20260809 19:20:59.956106 7570 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 4.1746
I20260809 19:20:59.956176 7570 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 4.1747
I20260809 19:20:59.956218 7570 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 4.1748
I20260809 19:20:59.956254 7570 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 4.1749
W20260809 19:20:59.956511 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:20:59.956491 7809 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1742, Last known committed idx: 1741, Time since last communication: 0.000s
W20260809 19:20:59.957551 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:20:59.957551 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:20:59.958815 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:21:00.402163 4039 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKf1XAf/build/release/bin/kudu with pid 7105
W20260809 19:21:00.416677 6681 meta_cache.cc:302] tablet 7c092d80045840deac3151a94e01de0b: replica 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997) has failed: Network error: recv got EOF from 127.3.241.195:36997 (error 108)
I20260809 19:21:00.417707 6381 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 5 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 4 index: 1399. Preceding OpId from leader: term: 5 index: 1742. (index mismatch)
I20260809 19:21:00.418321 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 5 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 4 index: 1535. Preceding OpId from leader: term: 5 index: 1742. (index mismatch)
W20260809 19:21:00.418911 5806 connection.cc:441] server connection from 127.3.241.195:53989 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.195:53989 (request call id 3) could send its response
W20260809 19:21:00.419119 6267 connection.cc:441] server connection from 127.3.241.195:35793 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.195:35793 (request call id 3) could send its response
W20260809 19:21:00.420783 5375 connection.cc:441] server connection from 127.3.241.195:47161 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.195:47161 (request call id 6) could send its response
W20260809 19:21:00.424789 7466 connection.cc:441] server connection from 127.3.241.195:36947 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.195:36947 (request call id 89) could send its response
I20260809 19:21:00.426162 4039 test_util.cc:268] Using random seed: 592215978
I20260809 19:21:00.529755 4039 test_util.cc:268] Using random seed: 592319567
I20260809 19:21:00.733978 4039 test_util.cc:268] Using random seed: 592523791
I20260809 19:21:01.037990 4039 test_util.cc:268] Using random seed: 592827804
I20260809 19:21:01.073855 7853 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:01.073979 7853 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:01.074294 7853 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:01.074553 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 6 candidate_status { last_received { term: 4 index: 1535 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:01.074538 6139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 6 candidate_status { last_received { term: 4 index: 1535 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:01.074784 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 6 candidate_status { last_received { term: 4 index: 1535 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:01.074898 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 5.
I20260809 19:21:01.074913 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 6 candidate_status { last_received { term: 4 index: 1535 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:01.074914 7569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 6 candidate_status { last_received { term: 4 index: 1535 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
W20260809 19:21:01.075021 5803 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260809 19:21:01.075280 5803 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595; no voters: 23183da38752444a9ff5525be50cd404, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
W20260809 19:21:01.075600 5803 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997): Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:21:01.075646 7853 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 19:21:01.442476 4039 test_util.cc:268] Using random seed: 593232291
I20260809 19:21:01.573565 7856 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 61427f2036ac4f96a9655aef910a7d7a)
I20260809 19:21:01.573684 7856 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:01.573985 7856 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:01.574257 6139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:01.574255 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:01.574270 7569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:01.574363 6139 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 5.
I20260809 19:21:01.574369 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 5.
W20260809 19:21:01.574466 6484 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:21:01.574680 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:01.574787 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:01.574908 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 5.
W20260809 19:21:01.574952 6484 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997): Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:21:01.575042 6484 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 61427f2036ac4f96a9655aef910a7d7a, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:01.575124 7856 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 19:21:01.575168 7856 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 61427f2036ac4f96a9655aef910a7d7a)
I20260809 19:21:01.575193 7856 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 5 FOLLOWER]: Advancing to term 6
I20260809 19:21:01.652336 7856 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:01.652854 7856 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 6 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:01.652895 6139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073"
I20260809 19:21:01.653021 6139 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 5 FOLLOWER]: Advancing to term 6
I20260809 19:21:01.653060 7569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:21:01.653230 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:01.653321 5918 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 5 FOLLOWER]: Advancing to term 6
W20260809 19:21:01.653576 6484 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997): Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:21:01.654062 6139 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 6.
I20260809 19:21:01.654039 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:01.654135 6381 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 5 FOLLOWER]: Advancing to term 6
I20260809 19:21:01.654210 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 6.
I20260809 19:21:01.654407 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:01.654935 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 6.
I20260809 19:21:01.655149 6484 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 61427f2036ac4f96a9655aef910a7d7a, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:01.655265 7856 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 19:21:01.655431 7856 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 6 LEADER]: Becoming Leader. State: Replica: 23183da38752444a9ff5525be50cd404, State: Running, Role: LEADER
I20260809 19:21:01.655561 7856 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1958, Committed index: 1958, Last appended: 5.1966, Last appended by leader: 1966, Current term: 6, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:01.656515 5040 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 reported cstate change: term changed from 5 to 6, leader changed from 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195) to 23183da38752444a9ff5525be50cd404 (127.3.241.199). New cstate: current_term: 6 leader_uuid: "23183da38752444a9ff5525be50cd404" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
I20260809 19:21:01.920989 7869 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 61427f2036ac4f96a9655aef910a7d7a)
I20260809 19:21:01.921104 7869 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:01.921334 7869 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:01.921640 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 6 candidate_status { last_received { term: 5 index: 1743 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:01.921774 5918 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 in current term 6: Already voted for candidate 23183da38752444a9ff5525be50cd404 in this term.
W20260809 19:21:01.921797 5375 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:21:01.921818 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 6 candidate_status { last_received { term: 5 index: 1743 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:01.921872 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 6 candidate_status { last_received { term: 5 index: 1743 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:01.921995 6381 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 in current term 6: Already voted for candidate 23183da38752444a9ff5525be50cd404 in this term.
I20260809 19:21:01.922006 6139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 6 candidate_status { last_received { term: 5 index: 1743 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:01.922117 6139 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 in current term 6: Already voted for candidate 23183da38752444a9ff5525be50cd404 in this term.
I20260809 19:21:01.922107 7569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 6 candidate_status { last_received { term: 5 index: 1743 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:01.922358 5375 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 7d4177500c7a4f41afce5190ff786506; no voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:01.922467 7869 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 5 FOLLOWER]: Advancing to term 6
W20260809 19:21:01.922552 5375 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997): Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:21:01.924721 7873 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 61427f2036ac4f96a9655aef910a7d7a)
I20260809 19:21:01.924808 7873 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:01.925056 7873 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:01.925181 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 6 candidate_status { last_received { term: 5 index: 1962 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:01.925222 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 6 candidate_status { last_received { term: 5 index: 1962 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:01.925345 5918 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 in current term 6: Already voted for candidate 23183da38752444a9ff5525be50cd404 in this term.
I20260809 19:21:01.925422 6139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 6 candidate_status { last_received { term: 5 index: 1962 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:01.925527 6139 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 in current term 6: Already voted for candidate 23183da38752444a9ff5525be50cd404 in this term.
I20260809 19:21:01.925583 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 6 candidate_status { last_received { term: 5 index: 1962 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:01.925679 6381 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 in current term 6: Already voted for candidate 23183da38752444a9ff5525be50cd404 in this term.
I20260809 19:21:01.925659 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 6 candidate_status { last_received { term: 5 index: 1962 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
W20260809 19:21:01.925804 7461 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:21:01.926012 7461 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: dd68273a4b314386ae0f5489e2f1d468; no voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad
I20260809 19:21:01.926100 7873 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 5 FOLLOWER]: Advancing to term 6
W20260809 19:21:01.926301 7461 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997): Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:21:01.963366 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.195:36997
--local_ip_for_outbound_sockets=127.3.241.195
--tserver_master_addrs=127.3.241.254:44125
--webserver_port=42257
--webserver_interface=127.3.241.195
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260809 19:21:01.963442 7869 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 19:21:01.963852 5479 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 6.
I20260809 19:21:01.964071 7873 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260809 19:21:02.050318 6484 consensus_peers.cc:487] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 -> Peer 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997): Couldn't send request to peer 61427f2036ac4f96a9655aef910a7d7a. Status: Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 19:21:02.053646 7877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:02.053859 7877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:02.053880 7877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:02.055004 7877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:02.055048 7877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.195
I20260809 19:21:02.056403 7877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.195:36997
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.3.241.195
--webserver_port=42257
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.7877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.195
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:02.056602 7877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:02.056860 7877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:02.062013 7884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:02.062067 7885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:02.062251 7889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:02.062295 7877 server_base.cc:1034] running on GCE node
I20260809 19:21:02.062522 7877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:02.062815 7877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:02.063998 7877 hybrid_clock.cc:648] HybridClock initialized: now 1786303262063982 us; error 44 us; skew 500 ppm
I20260809 19:21:02.068878 7877 webserver.cc:460] Webserver started at http://127.3.241.195:42257/ using document root <none> and password file <none>
I20260809 19:21:02.069177 7877 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:02.069259 7877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:02.070832 7877 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:21:02.071610 7898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:02.071836 7877 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:21:02.071947 7877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "61427f2036ac4f96a9655aef910a7d7a"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:21:02.072378 7877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:02.075889 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 6 FOLLOWER]: Refusing update from remote peer 23183da38752444a9ff5525be50cd404: Log matching property violated. Preceding OpId in replica: term: 4 index: 1535. Preceding OpId from leader: term: 6 index: 1967. (index mismatch)
I20260809 19:21:02.076249 7856 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1967, Last known committed idx: 1534, Time since last communication: 0.000s
I20260809 19:21:02.080679 7877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:02.081028 7877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:02.081189 7877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:02.081449 7877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:02.081982 7918 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:21:02.083690 7877 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:21:02.083758 7877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:21:02.083808 7877 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:21:02.084735 7877 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:21:02.084774 7877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 19:21:02.084877 7918 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Bootstrap starting.
I20260809 19:21:02.093112 7877 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.195:36997
I20260809 19:21:02.093629 7877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260809 19:21:02.100812 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 7877
I20260809 19:21:02.103214 8086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.195:36997 every 8 connection(s)
I20260809 19:21:02.113157 8087 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:21:02.113401 8087 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:02.113718 8087 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:21:02.114503 5040 ts_manager.cc:194] Re-registered known tserver with Master: 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997)
I20260809 19:21:02.115006 5040 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.195:57557
I20260809 19:21:02.123988 5479 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 6 FOLLOWER]: Refusing update from remote peer 23183da38752444a9ff5525be50cd404: Log matching property violated. Preceding OpId in replica: term: 5 index: 1743. Preceding OpId from leader: term: 6 index: 1967. (index mismatch)
I20260809 19:21:02.124328 7856 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1967, Last known committed idx: 1742, Time since last communication: 0.000s
I20260809 19:21:02.130745 7918 log.cc:824] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:02.199262 6139 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 6 FOLLOWER]: Refusing update from remote peer 23183da38752444a9ff5525be50cd404: Log matching property violated. Preceding OpId in replica: term: 5 index: 1966. Preceding OpId from leader: term: 6 index: 1967. (index mismatch)
I20260809 19:21:02.199715 7856 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [LEADER]: Connected to new peer: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1967, Last known committed idx: 1958, Time since last communication: 0.000s
I20260809 19:21:02.202229 7569 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 6 FOLLOWER]: Refusing update from remote peer 23183da38752444a9ff5525be50cd404: Log matching property violated. Preceding OpId in replica: term: 5 index: 1962. Preceding OpId from leader: term: 6 index: 1967. (index mismatch)
I20260809 19:21:02.203087 7856 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1967, Last known committed idx: 1955, Time since last communication: 0.000s
I20260809 19:21:02.204124 8119 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1953, attempt_no=1}) took 1778 ms. Trace:
I20260809 19:21:02.204650 8119 rpcz_store.cc:276] 0809 19:21:00.425126 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:00.425405 (+ 279us) service_pool.cc:227] Handling call
0809 19:21:02.204112 (+1778707us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:21:02.204603 8123 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1955, attempt_no=1}) took 1779 ms. Trace:
I20260809 19:21:02.205250 8123 rpcz_store.cc:276] 0809 19:21:00.425109 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:00.425260 (+ 151us) service_pool.cc:227] Handling call
0809 19:21:02.204597 (+1779337us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:21:02.204430 8120 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1954, attempt_no=1}) took 1779 ms. Trace:
I20260809 19:21:02.205474 8120 rpcz_store.cc:276] 0809 19:21:00.425173 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:00.425323 (+ 150us) service_pool.cc:227] Handling call
0809 19:21:02.204423 (+1779100us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:21:02.204881 8126 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1956, attempt_no=1}) took 1779 ms. Trace:
I20260809 19:21:02.205627 8126 rpcz_store.cc:276] 0809 19:21:00.425159 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:00.425302 (+ 143us) service_pool.cc:227] Handling call
0809 19:21:02.204873 (+1779571us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:21:02.205755 8126 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1960, attempt_no=1}) took 1780 ms. Trace:
I20260809 19:21:02.205787 8126 rpcz_store.cc:276] 0809 19:21:00.425142 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:00.425420 (+ 278us) service_pool.cc:227] Handling call
0809 19:21:02.205752 (+1780332us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:21:02.205864 8120 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1958, attempt_no=1}) took 1780 ms. Trace:
I20260809 19:21:02.205896 8120 rpcz_store.cc:276] 0809 19:21:00.425239 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:00.425383 (+ 144us) service_pool.cc:227] Handling call
0809 19:21:02.205863 (+1780480us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:21:02.205964 8119 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1957, attempt_no=1}) took 1780 ms. Trace:
I20260809 19:21:02.205991 8119 rpcz_store.cc:276] 0809 19:21:00.425212 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:00.425344 (+ 132us) service_pool.cc:227] Handling call
0809 19:21:02.205962 (+1780618us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:21:02.206061 8123 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33178 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=1959, attempt_no=1}) took 1780 ms. Trace:
I20260809 19:21:02.206089 8123 rpcz_store.cc:276] 0809 19:21:00.425190 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:00.425365 (+ 175us) service_pool.cc:227] Handling call
0809 19:21:02.206060 (+1780695us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 19:21:02.206386 6082 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:02.206573 6080 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:02.207038 6080 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:02.207038 6082 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:02.207113 6070 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:02.207270 6074 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:02.207414 6076 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:02.207537 6083 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:33178: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
I20260809 19:21:02.332381 7918 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1973 overwritten=7 applied=1958 ignored=0} inserts{seen=19530 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260809 19:21:02.333021 7918 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Bootstrap complete.
I20260809 19:21:02.334580 7918 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Time spent bootstrapping tablet: real 0.250s user 0.177s sys 0.054s
I20260809 19:21:02.336931 7918 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 5 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:02.338275 7918 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61427f2036ac4f96a9655aef910a7d7a, State: Initialized, Role: FOLLOWER
I20260809 19:21:02.338510 7918 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1958, Last appended: 5.1966, Last appended by leader: 1966, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:02.339026 7918 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 19:21:02.339215 8087 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
I20260809 19:21:02.825475 6381 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 6 FOLLOWER]: Refusing update from remote peer 23183da38752444a9ff5525be50cd404: Log matching property violated. Preceding OpId in replica: term: 4 index: 1399. Preceding OpId from leader: term: 6 index: 1967. (index mismatch)
I20260809 19:21:02.932629 4039 test_util.cc:268] Using random seed: 594722439
I20260809 19:21:03.238884 4039 test_util.cc:268] Using random seed: 595028695
I20260809 19:21:03.550308 8168 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:03.550418 8168 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:03.550695 8168 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545)
I20260809 19:21:03.550904 6146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 7 candidate_status { last_received { term: 4 index: 1399 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:03.550952 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 7 candidate_status { last_received { term: 4 index: 1399 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:03.550966 7570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 7 candidate_status { last_received { term: 4 index: 1399 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:03.551245 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 7 candidate_status { last_received { term: 4 index: 1399 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:03.551620 6267 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: d4c4b93755eb4aa48c04795ad08e09ad; no voters: 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:03.551743 8168 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 19:21:03.554661 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 7 candidate_status { last_received { term: 4 index: 1399 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:03.554834 8006 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4c4b93755eb4aa48c04795ad08e09ad for term 7 because replica has last-logged OpId of term: 5 index: 1966, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 1399.
I20260809 19:21:03.644198 4039 test_util.cc:268] Using random seed: 595434009
I20260809 19:21:03.864971 8172 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:03.865182 8172 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:03.865669 8172 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:03.870265 7570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:03.870747 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:03.870950 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:03.871060 6381 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a in current term 6: Already voted for candidate 23183da38752444a9ff5525be50cd404 in this term.
I20260809 19:21:03.871563 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:03.871909 7902 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a; no voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:03.872073 8172 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 5 FOLLOWER]: Advancing to term 6
I20260809 19:21:03.872519 6146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:03.873481 8172 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 19:21:03.876644 8195 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 23183da38752444a9ff5525be50cd404)
I20260809 19:21:03.876824 8196 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 23183da38752444a9ff5525be50cd404)
I20260809 19:21:03.876751 8195 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:03.876909 8196 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:03.877146 8195 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:03.877297 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:03.877511 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:03.877534 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:03.877636 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:03.877724 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 6.
I20260809 19:21:03.877795 8196 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:03.877919 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:03.877936 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:03.877964 7570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:03.878007 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 6.
I20260809 19:21:03.878036 7570 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 6.
I20260809 19:21:03.878052 6146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:03.878134 6146 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 6.
I20260809 19:21:03.881022 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:03.881107 8006 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 6.
I20260809 19:21:03.881268 7461 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506
I20260809 19:21:03.881366 8195 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 19:21:03.881407 8195 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 23183da38752444a9ff5525be50cd404)
I20260809 19:21:03.881429 8195 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 6 FOLLOWER]: Advancing to term 7
I20260809 19:21:03.881443 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:03.881534 8006 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 6.
I20260809 19:21:03.881716 6034 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506
I20260809 19:21:03.881834 8196 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 19:21:03.881875 8196 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 23183da38752444a9ff5525be50cd404)
I20260809 19:21:03.881902 8196 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 6 FOLLOWER]: Advancing to term 7
I20260809 19:21:03.882248 8195 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:03.882447 8195 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 7 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:03.882550 6146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073"
I20260809 19:21:03.882659 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:03.882728 6381 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 6 FOLLOWER]: Advancing to term 7
I20260809 19:21:03.882707 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:03.882709 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:03.882769 8006 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 6 FOLLOWER]: Advancing to term 7
I20260809 19:21:03.882866 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:03.883092 8196 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:03.883215 6146 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 in current term 7: Already voted for candidate 757ec8723e924451a89abb5cc62c1073 in this term.
I20260809 19:21:03.883376 8196 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 7 election: Requested vote from peers 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:03.883481 8006 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 7.
I20260809 19:21:03.883481 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 7.
I20260809 19:21:03.883502 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:03.883548 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:03.883683 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:03.883754 6381 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 757ec8723e924451a89abb5cc62c1073 in current term 7: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:03.883806 7570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:21:03.883900 7570 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 757ec8723e924451a89abb5cc62c1073 in current term 7: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:03.884023 6034 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 757ec8723e924451a89abb5cc62c1073; no voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:03.884120 8196 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 19:21:03.884140 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:03.884249 8006 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 757ec8723e924451a89abb5cc62c1073 in current term 7: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:04.149487 4039 test_util.cc:268] Using random seed: 595939296
I20260809 19:21:04.327358 8204 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 23183da38752444a9ff5525be50cd404)
I20260809 19:21:04.327503 8204 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:04.327910 8204 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:04.328043 6146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 7 candidate_status { last_received { term: 6 index: 2042 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:04.328181 6146 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 7: Already voted for candidate 757ec8723e924451a89abb5cc62c1073 in this term.
I20260809 19:21:04.328156 7570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 7 candidate_status { last_received { term: 6 index: 2042 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:04.328276 7570 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 7: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:04.328471 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 7 candidate_status { last_received { term: 6 index: 2042 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:04.328508 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 7 candidate_status { last_received { term: 6 index: 2042 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:04.328655 6381 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 7: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:04.328770 5803 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595; no voters: 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:04.328882 8204 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 6 FOLLOWER]: Advancing to term 7
I20260809 19:21:04.329892 8204 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 19:21:04.331575 8207 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 23183da38752444a9ff5525be50cd404)
I20260809 19:21:04.331661 8207 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:04.331975 8207 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:04.332078 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 7 candidate_status { last_received { term: 6 index: 2198 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:04.332216 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 7.
I20260809 19:21:04.332260 6146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 7 candidate_status { last_received { term: 6 index: 2198 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:04.332324 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 7 candidate_status { last_received { term: 6 index: 2042 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:04.332396 6146 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 in current term 7: Already voted for candidate 757ec8723e924451a89abb5cc62c1073 in this term.
I20260809 19:21:04.332444 8006 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 7: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:04.332584 7570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 7 candidate_status { last_received { term: 6 index: 2198 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:04.332696 7570 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 in current term 7: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:04.333000 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 7 candidate_status { last_received { term: 6 index: 2198 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:04.333120 6381 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 in current term 7: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:04.335479 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 7 candidate_status { last_received { term: 6 index: 2198 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:04.335561 8006 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 in current term 7: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:04.335698 5375 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506; no voters: 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:04.335805 8207 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 6 FOLLOWER]: Advancing to term 7
I20260809 19:21:04.336520 8207 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 19:21:04.754980 4039 test_util.cc:268] Using random seed: 596544784
W20260809 19:21:05.446100 6270 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.241.199:41771 timed out after 1.895s (SENT)
I20260809 19:21:05.460407 4039 test_util.cc:268] Using random seed: 597250219
W20260809 19:21:05.538805 6039 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.3.241.199:41771 timed out after 1.660s (SENT) [suppressed 2 similar messages]
W20260809 19:21:05.626121 7903 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.3.241.199:41771 for RPC RequestConsensusVote timed out after 1.759s (ON_OUTBOUND_QUEUE)
I20260809 19:21:05.859370 8218 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:05.859480 8218 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:05.859730 8218 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:05.860011 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 8 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:05.860001 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 8 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:05.859999 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 8 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:05.860001 7570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 8 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:05.860112 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 7.
I20260809 19:21:05.860112 8006 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 7.
I20260809 19:21:05.860117 7570 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 7.
I20260809 19:21:05.860117 5479 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 7.
I20260809 19:21:05.860293 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 8 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:05.860360 6039 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, dd68273a4b314386ae0f5489e2f1d468; no voters:
I20260809 19:21:05.860411 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 7.
I20260809 19:21:05.860455 8218 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 19:21:05.860489 8218 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:21:05.860523 8218 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 7 FOLLOWER]: Advancing to term 8
I20260809 19:21:05.861322 8218 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:05.861522 8218 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 8 election: Requested vote from peers 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:05.861672 7570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 8 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:21:05.861719 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 8 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:05.861743 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 8 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:05.861778 7570 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 7 FOLLOWER]: Advancing to term 8
I20260809 19:21:05.861797 8006 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 7 FOLLOWER]: Advancing to term 8
I20260809 19:21:05.861802 6381 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 7 FOLLOWER]: Advancing to term 8
I20260809 19:21:05.861786 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 8 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:05.861889 5918 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 7 FOLLOWER]: Advancing to term 8
I20260809 19:21:05.862582 8006 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 8.
I20260809 19:21:05.862582 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 8.
I20260809 19:21:05.862686 7570 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 8.
I20260809 19:21:05.862766 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 8.
I20260809 19:21:05.862780 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 8 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:05.862875 5479 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 7 FOLLOWER]: Advancing to term 8
I20260809 19:21:05.862874 6034 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters:
I20260809 19:21:05.862957 8218 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 19:21:05.863121 8218 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 8 LEADER]: Becoming Leader. State: Replica: 757ec8723e924451a89abb5cc62c1073, State: Running, Role: LEADER
I20260809 19:21:05.863195 8218 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2198, Committed index: 2198, Last appended: 6.2206, Last appended by leader: 2206, Current term: 8, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:05.863767 5479 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 8.
I20260809 19:21:05.863943 5040 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 reported cstate change: term changed from 6 to 8, leader changed from 23183da38752444a9ff5525be50cd404 (127.3.241.199) to 757ec8723e924451a89abb5cc62c1073 (127.3.241.197). New cstate: current_term: 8 leader_uuid: "757ec8723e924451a89abb5cc62c1073" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
W20260809 19:21:05.904668 8096 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:02.827690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.241.199:36065 (local address 127.3.241.195:36997)
0809 19:21:02.828831 (+ 1141us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:02.828833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:02.828845 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:21:02.828899 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:02.828901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:02.828917 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:02.829018 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:05.904435 (+3075417us) negotiation.cc:327] Negotiation complete: Timed out: Server connection negotiation failed: server connection from 127.3.241.199:36065: received 0 of 4 requested bytes
Metrics: {"server-negotiator.queue_time_us":1115}
I20260809 19:21:06.166313 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:06.166460 6584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 7 candidate_status { last_received { term: 4 index: 1399 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:06.166658 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 7 candidate_status { last_received { term: 4 index: 1399 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:06.166803 6589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:06.166894 6591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:06.166976 6146 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 8 LEADER]: Rejecting Update request from peer 23183da38752444a9ff5525be50cd404 for earlier term 6. Current term is 8. Ops: []
I20260809 19:21:06.167008 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:06.167461 8152 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: INVALID_TERM, Last received: 6.2206, Next index: 2207, Last known committed idx: 2198, Time since last communication: 0.000s
I20260809 19:21:06.167470 7462 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506
I20260809 19:21:06.167544 8152 consensus_queue.cc:1225] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [LEADER]: Peer 757ec8723e924451a89abb5cc62c1073 log is divergent from this leader: its last log entry 8.2207 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2198
I20260809 19:21:06.167627 8152 raft_consensus.cc:3032] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 19:21:06.167651 8152 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 23183da38752444a9ff5525be50cd404, State: Running, Role: LEADER
I20260809 19:21:06.167730 8228 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 19:21:06.167707 8152 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2198, Committed index: 2198, Last appended: 6.2206, Last appended by leader: 2206, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:06.167860 8152 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 6 FOLLOWER]: Advancing to term 8
W20260809 19:21:06.167655 7857 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:02.535897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.241.195:36997 (local address 127.3.241.199:36065)
0809 19:21:02.535962 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:02.535975 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:02.536046 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:06.167581 (+3631535us) negotiation.cc:327] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.3.241.195:36997: received 0 of 25 requested bytes
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 19:21:06.166501 6486 connection.cc:232] RPC call timeout handler was delayed by -3.0050594790372998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260809 19:21:06.168810 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 7 candidate_status { last_received { term: 6 index: 2198 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
W20260809 19:21:06.166474 6484 connection.cc:232] RPC call timeout handler was delayed by -3.0645196239929646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260809 19:21:06.169062 6484 connection.cc:232] RPC call timeout handler was delayed by -2.8964026300236583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260809 19:21:06.169078 6592 raft_consensus.cc:2354] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 for earlier term 7. Current term is 8.
W20260809 19:21:06.169086 6484 connection.cc:232] RPC call timeout handler was delayed by -2.7307486028876156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260809 19:21:06.169374 6571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:06.169459 6571 raft_consensus.cc:2354] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 757ec8723e924451a89abb5cc62c1073 for earlier term 7. Current term is 8.
I20260809 19:21:06.169497 6580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:06.169519 6584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 8 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:06.169556 7570 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 8 FOLLOWER]: Rejecting Update request from peer 23183da38752444a9ff5525be50cd404 for earlier term 6. Current term is 8. Ops: []
I20260809 19:21:06.169557 6582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 8 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:06.169505 6592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:06.169567 6580 raft_consensus.cc:2354] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 757ec8723e924451a89abb5cc62c1073 for earlier term 7. Current term is 8.
I20260809 19:21:06.169535 6578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 7 candidate_status { last_received { term: 6 index: 2206 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:06.169763 6578 raft_consensus.cc:2354] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 757ec8723e924451a89abb5cc62c1073 for earlier term 7. Current term is 8.
I20260809 19:21:06.169864 6591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 7 candidate_status { last_received { term: 6 index: 2042 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:06.169902 5377 leader_election.cc:398] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771) with higher term. Message: Invalid argument: T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 for earlier term 7. Current term is 8.
I20260809 19:21:06.169951 6591 raft_consensus.cc:2354] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for earlier term 7. Current term is 8.
I20260809 19:21:06.170015 6589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 7 candidate_status { last_received { term: 6 index: 2042 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:06.170066 6589 raft_consensus.cc:2354] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for earlier term 7. Current term is 8.
I20260809 19:21:06.170228 5804 leader_election.cc:398] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771) with higher term. Message: Invalid argument: T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for earlier term 7. Current term is 8.
I20260809 19:21:06.170385 6039 leader_election.cc:398] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771) with higher term. Message: Invalid argument: T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 757ec8723e924451a89abb5cc62c1073 for earlier term 7. Current term is 8.
I20260809 19:21:06.170593 6039 leader_election.cc:398] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 7 election: Vote denied by peer 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771) with higher term. Message: Invalid argument: T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 757ec8723e924451a89abb5cc62c1073 for earlier term 7. Current term is 8.
I20260809 19:21:06.173156 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 6 candidate_status { last_received { term: 5 index: 1966 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:06.173249 6586 raft_consensus.cc:2354] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a for earlier term 6. Current term is 8.
I20260809 19:21:06.173447 7903 leader_election.cc:398] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 6 pre-election: Vote denied by peer 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771) with higher term. Message: Invalid argument: T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a for earlier term 6. Current term is 8.
I20260809 19:21:06.175472 8006 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 8 FOLLOWER]: Rejecting Update request from peer 23183da38752444a9ff5525be50cd404 for earlier term 6. Current term is 8. Ops: [6.1967-6.2206]
I20260809 19:21:06.239984 5479 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 8 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 6 index: 2198. Preceding OpId from leader: term: 8 index: 2207. (index mismatch)
I20260809 19:21:06.240356 8218 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2207, Last known committed idx: 2191, Time since last communication: 0.000s
I20260809 19:21:06.305032 7570 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 8 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 6 index: 2206. Preceding OpId from leader: term: 8 index: 2207. (index mismatch)
I20260809 19:21:06.305418 8218 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2207, Last known committed idx: 2198, Time since last communication: 0.000s
I20260809 19:21:06.355451 8006 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 8 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 5 index: 1966. Preceding OpId from leader: term: 8 index: 2207. (index mismatch)
I20260809 19:21:06.355822 8218 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2207, Last known committed idx: 1958, Time since last communication: 0.000s
I20260809 19:21:06.363291 8254 mvcc.cc:205] Tried to move back new op lower bound from 7316698162599788544 to 7316698159742177280. Current Snapshot: MvccSnapshot[applied={T|T < 7316698162084048896}]
I20260809 19:21:06.365223 6586 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 6 index: 2206. Preceding OpId from leader: term: 8 index: 2207. (index mismatch)
I20260809 19:21:06.365901 8218 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2207, Last known committed idx: 2198, Time since last communication: 0.000s
I20260809 19:21:06.368181 8279 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2192, attempt_no=0}) took 3994 ms. Trace:
I20260809 19:21:06.368373 8279 rpcz_store.cc:276] 0809 19:21:02.373774 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:02.373798 (+ 24us) service_pool.cc:227] Handling call
0809 19:21:06.368147 (+3994349us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:02.373884 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:02.373892 (+ 8us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:02.373892 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:21:02.373893 (+ 1us) tablet.cc:660] Decoding operations
0809 19:21:02.373909 (+ 16us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:02.373911 (+ 2us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:02.373912 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:02.373923 (+ 11us) tablet.cc:699] Row locks acquired
0809 19:21:02.373923 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:21:02.373927 (+ 4us) write_op.cc:270] Start()
0809 19:21:02.373930 (+ 3us) write_op.cc:276] Timestamp: P: 1786303262373847 usec, L: 0
0809 19:21:02.373930 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:21:02.373956 (+ 26us) log.cc:842] Serialized 696 byte log entry
0809 19:21:06.366194 (+3992238us) op_driver.cc:444] REPLICATION: finished
0809 19:21:06.368060 (+ 1866us) write_op.cc:301] APPLY: starting
0809 19:21:06.368064 (+ 4us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:06.368069 (+ 5us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:06.368073 (+ 4us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:06.368094 (+ 21us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:06.368098 (+ 4us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:06.368098 (+ 0us) write_op.cc:312] APPLY: finished
0809 19:21:06.368113 (+ 15us) log.cc:842] Serialized 77 byte log entry
0809 19:21:06.368124 (+ 11us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:06.368129 (+ 5us) write_op.cc:454] Released schema lock
0809 19:21:06.368129 (+ 0us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1857,"num_ops":10,"prepare.queue_time_us":19,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":368,"raft.queue_time_us":349,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"replication_time_us":3992260,"spinlock_wait_cycles":6272,"thread_start_us":59,"threads_started":1}]]}
I20260809 19:21:06.369058 8279 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2193, attempt_no=0}) took 3994 ms. Trace:
I20260809 19:21:06.369203 8279 rpcz_store.cc:276] 0809 19:21:02.374134 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:02.374168 (+ 34us) service_pool.cc:227] Handling call
0809 19:21:06.369053 (+3994885us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:02.374257 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:02.374264 (+ 7us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:02.374264 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:21:02.374264 (+ 0us) tablet.cc:660] Decoding operations
0809 19:21:02.374274 (+ 10us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:02.374275 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:02.374276 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:02.374283 (+ 7us) tablet.cc:699] Row locks acquired
0809 19:21:02.374283 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:21:02.374288 (+ 5us) write_op.cc:270] Start()
0809 19:21:02.374290 (+ 2us) write_op.cc:276] Timestamp: P: 1786303262374207 usec, L: 0
0809 19:21:02.374290 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:21:02.374312 (+ 22us) log.cc:842] Serialized 696 byte log entry
0809 19:21:06.366262 (+3991950us) op_driver.cc:444] REPLICATION: finished
0809 19:21:06.368972 (+ 2710us) write_op.cc:301] APPLY: starting
0809 19:21:06.368974 (+ 2us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:06.368978 (+ 4us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:06.368979 (+ 1us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:06.369007 (+ 28us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:06.369011 (+ 4us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:06.369012 (+ 1us) write_op.cc:312] APPLY: finished
0809 19:21:06.369023 (+ 11us) log.cc:842] Serialized 77 byte log entry
0809 19:21:06.369034 (+ 11us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:06.369038 (+ 4us) write_op.cc:454] Released schema lock
0809 19:21:06.369039 (+ 1us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2703,"num_ops":10,"prepare.queue_time_us":28,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":80,"raft.queue_time_us":15,"raft.run_cpu_time_us":2,"raft.run_wall_time_us":3,"replication_time_us":3991969,"thread_start_us":33,"threads_started":1}]]}
I20260809 19:21:06.369614 8279 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2194, attempt_no=0}) took 3995 ms. Trace:
I20260809 19:21:06.369737 8279 rpcz_store.cc:276] 0809 19:21:02.374182 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:02.374203 (+ 21us) service_pool.cc:227] Handling call
0809 19:21:06.369611 (+3995408us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:02.374340 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:02.374346 (+ 6us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:02.374346 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:21:02.374346 (+ 0us) tablet.cc:660] Decoding operations
0809 19:21:02.374362 (+ 16us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:02.374363 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:02.374364 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:02.374369 (+ 5us) tablet.cc:699] Row locks acquired
0809 19:21:02.374369 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:21:02.374371 (+ 2us) write_op.cc:270] Start()
0809 19:21:02.374375 (+ 4us) write_op.cc:276] Timestamp: P: 1786303262374291 usec, L: 0
0809 19:21:02.374375 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:21:02.374386 (+ 11us) log.cc:842] Serialized 696 byte log entry
0809 19:21:06.366302 (+3991916us) op_driver.cc:444] REPLICATION: finished
0809 19:21:06.369548 (+ 3246us) write_op.cc:301] APPLY: starting
0809 19:21:06.369550 (+ 2us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:06.369553 (+ 3us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:06.369555 (+ 2us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:06.369573 (+ 18us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:06.369575 (+ 2us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:06.369575 (+ 0us) write_op.cc:312] APPLY: finished
0809 19:21:06.369585 (+ 10us) log.cc:842] Serialized 77 byte log entry
0809 19:21:06.369595 (+ 10us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:06.369598 (+ 3us) write_op.cc:454] Released schema lock
0809 19:21:06.369599 (+ 1us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3240,"num_ops":10,"prepare.queue_time_us":92,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":59,"raft.queue_time_us":62,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":3,"replication_time_us":3991924,"thread_start_us":77,"threads_started":1}]]}
I20260809 19:21:06.370147 8279 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2195, attempt_no=0}) took 3995 ms. Trace:
W20260809 19:21:06.370256 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:06.369733 6527 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:21:06.370290 8279 rpcz_store.cc:276] 0809 19:21:02.374277 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:02.374299 (+ 22us) service_pool.cc:227] Handling call
0809 19:21:06.370143 (+3995844us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:02.374402 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:02.374406 (+ 4us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:02.374407 (+ 1us) write_op.cc:435] Acquired schema lock
0809 19:21:02.374407 (+ 0us) tablet.cc:660] Decoding operations
0809 19:21:02.374412 (+ 5us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:02.374412 (+ 0us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:02.374413 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:02.374417 (+ 4us) tablet.cc:699] Row locks acquired
0809 19:21:02.374418 (+ 1us) write_op.cc:260] PREPARE: finished
0809 19:21:02.374419 (+ 1us) write_op.cc:270] Start()
0809 19:21:02.374421 (+ 2us) write_op.cc:276] Timestamp: P: 1786303262374340 usec, L: 0
0809 19:21:02.374421 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:21:02.374429 (+ 8us) log.cc:842] Serialized 686 byte log entry
0809 19:21:06.366386 (+3991957us) op_driver.cc:444] REPLICATION: finished
0809 19:21:06.370078 (+ 3692us) write_op.cc:301] APPLY: starting
0809 19:21:06.370079 (+ 1us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:06.370084 (+ 5us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:06.370085 (+ 1us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:06.370107 (+ 22us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:06.370109 (+ 2us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:06.370109 (+ 0us) write_op.cc:312] APPLY: finished
0809 19:21:06.370118 (+ 9us) log.cc:842] Serialized 77 byte log entry
0809 19:21:06.370129 (+ 11us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:06.370133 (+ 4us) write_op.cc:454] Released schema lock
0809 19:21:06.370133 (+ 0us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3687,"num_ops":10,"prepare.queue_time_us":74,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":41,"raft.queue_time_us":28,"raft.run_cpu_time_us":2,"raft.run_wall_time_us":1,"replication_time_us":3991962,"thread_start_us":76,"threads_started":1}]]}
I20260809 19:21:06.370537 8279 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2196, attempt_no=0}) took 3995 ms. Trace:
I20260809 19:21:06.370639 8279 rpcz_store.cc:276] 0809 19:21:02.374671 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:02.374696 (+ 25us) service_pool.cc:227] Handling call
0809 19:21:06.370534 (+3995838us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:02.374808 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:02.374813 (+ 5us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:02.374813 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:21:02.374813 (+ 0us) tablet.cc:660] Decoding operations
0809 19:21:02.374824 (+ 11us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:02.374825 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:02.374826 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:02.374835 (+ 9us) tablet.cc:699] Row locks acquired
0809 19:21:02.374836 (+ 1us) write_op.cc:260] PREPARE: finished
0809 19:21:02.374838 (+ 2us) write_op.cc:270] Start()
0809 19:21:02.374839 (+ 1us) write_op.cc:276] Timestamp: P: 1786303262374758 usec, L: 0
0809 19:21:02.374840 (+ 1us) op_driver.cc:331] REPLICATION: starting
0809 19:21:02.374862 (+ 22us) log.cc:842] Serialized 696 byte log entry
0809 19:21:06.366468 (+3991606us) op_driver.cc:444] REPLICATION: finished
0809 19:21:06.370474 (+ 4006us) write_op.cc:301] APPLY: starting
0809 19:21:06.370475 (+ 1us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:06.370479 (+ 4us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:06.370481 (+ 2us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:06.370499 (+ 18us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:06.370501 (+ 2us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:06.370501 (+ 0us) write_op.cc:312] APPLY: finished
0809 19:21:06.370510 (+ 9us) log.cc:842] Serialized 77 byte log entry
0809 19:21:06.370520 (+ 10us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:06.370524 (+ 4us) write_op.cc:454] Released schema lock
0809 19:21:06.370524 (+ 0us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4001,"num_ops":10,"prepare.queue_time_us":51,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":98,"raft.queue_time_us":65,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":2,"replication_time_us":3991625,"spinlock_wait_cycles":3840}]]}
I20260809 19:21:06.370997 8279 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2198, attempt_no=0}) took 3996 ms. Trace:
I20260809 19:21:06.371045 8279 rpcz_store.cc:276] 0809 19:21:02.374714 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:02.374738 (+ 24us) service_pool.cc:227] Handling call
0809 19:21:06.370992 (+3996254us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:02.374909 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:02.374913 (+ 4us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:02.374914 (+ 1us) write_op.cc:435] Acquired schema lock
0809 19:21:02.374914 (+ 0us) tablet.cc:660] Decoding operations
0809 19:21:02.374931 (+ 17us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:02.374931 (+ 0us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:02.374932 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:02.375009 (+ 77us) tablet.cc:699] Row locks acquired
0809 19:21:02.375009 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:21:02.375012 (+ 3us) write_op.cc:270] Start()
0809 19:21:02.375014 (+ 2us) write_op.cc:276] Timestamp: P: 1786303262374932 usec, L: 0
0809 19:21:02.375014 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:21:02.375026 (+ 12us) log.cc:842] Serialized 696 byte log entry
0809 19:21:06.366473 (+3991447us) op_driver.cc:444] REPLICATION: finished
0809 19:21:06.370915 (+ 4442us) write_op.cc:301] APPLY: starting
0809 19:21:06.370916 (+ 1us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:06.370920 (+ 4us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:06.370922 (+ 2us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:06.370951 (+ 29us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:06.370954 (+ 3us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:06.370954 (+ 0us) write_op.cc:312] APPLY: finished
0809 19:21:06.370964 (+ 10us) log.cc:842] Serialized 77 byte log entry
0809 19:21:06.370977 (+ 13us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:06.370981 (+ 4us) write_op.cc:454] Released schema lock
0809 19:21:06.370982 (+ 1us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4435,"num_ops":10,"prepare.queue_time_us":135,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":140,"raft.queue_time_us":35,"raft.run_cpu_time_us":2,"raft.run_wall_time_us":2,"replication_time_us":3991456}]]}
I20260809 19:21:06.371448 8282 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2197, attempt_no=0}) took 3996 ms. Trace:
W20260809 19:21:06.371570 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:21:06.371685 8284 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2199, attempt_no=0}) took 3996 ms. Trace:
W20260809 19:21:06.371794 6527 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:06.371726 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:21:06.371790 8284 rpcz_store.cc:276] 0809 19:21:02.374732 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:02.374770 (+ 38us) service_pool.cc:227] Handling call
0809 19:21:06.371680 (+3996910us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:02.375116 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:02.375120 (+ 4us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:02.375120 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:21:02.375121 (+ 1us) tablet.cc:660] Decoding operations
0809 19:21:02.375129 (+ 8us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:02.375130 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:02.375130 (+ 0us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:02.375135 (+ 5us) tablet.cc:699] Row locks acquired
0809 19:21:02.375135 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:21:02.375137 (+ 2us) write_op.cc:270] Start()
0809 19:21:02.375139 (+ 2us) write_op.cc:276] Timestamp: P: 1786303262375057 usec, L: 0
0809 19:21:02.375139 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:21:02.375151 (+ 12us) log.cc:842] Serialized 686 byte log entry
0809 19:21:06.366481 (+3991330us) op_driver.cc:444] REPLICATION: finished
0809 19:21:06.371604 (+ 5123us) write_op.cc:301] APPLY: starting
0809 19:21:06.371606 (+ 2us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:06.371610 (+ 4us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:06.371620 (+ 10us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:06.371638 (+ 18us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:06.371640 (+ 2us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:06.371641 (+ 1us) write_op.cc:312] APPLY: finished
0809 19:21:06.371653 (+ 12us) log.cc:842] Serialized 77 byte log entry
0809 19:21:06.371665 (+ 12us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:06.371669 (+ 4us) write_op.cc:454] Released schema lock
0809 19:21:06.371669 (+ 0us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":5114,"num_ops":10,"prepare.queue_time_us":318,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":19,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":2,"replication_time_us":3991339}]]}
W20260809 19:21:06.372051 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:21:06.371744 8282 rpcz_store.cc:276] 0809 19:21:02.374693 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:02.374713 (+ 20us) service_pool.cc:227] Handling call
0809 19:21:06.371441 (+3996728us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:02.375053 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:02.375058 (+ 5us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:02.375058 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:21:02.375058 (+ 0us) tablet.cc:660] Decoding operations
0809 19:21:02.375068 (+ 10us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:02.375069 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:02.375069 (+ 0us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:02.375079 (+ 10us) tablet.cc:699] Row locks acquired
0809 19:21:02.375079 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:21:02.375082 (+ 3us) write_op.cc:270] Start()
0809 19:21:02.375083 (+ 1us) write_op.cc:276] Timestamp: P: 1786303262375002 usec, L: 0
0809 19:21:02.375084 (+ 1us) op_driver.cc:331] REPLICATION: starting
0809 19:21:02.375096 (+ 12us) log.cc:842] Serialized 696 byte log entry
0809 19:21:06.366477 (+3991381us) op_driver.cc:444] REPLICATION: finished
0809 19:21:06.371357 (+ 4880us) write_op.cc:301] APPLY: starting
0809 19:21:06.371359 (+ 2us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:06.371364 (+ 5us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:06.371376 (+ 12us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:06.371395 (+ 19us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:06.371397 (+ 2us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:06.371398 (+ 1us) write_op.cc:312] APPLY: finished
0809 19:21:06.371410 (+ 12us) log.cc:842] Serialized 77 byte log entry
0809 19:21:06.371424 (+ 14us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:06.371428 (+ 4us) write_op.cc:454] Released schema lock
0809 19:21:06.371429 (+ 1us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4871,"num_ops":10,"prepare.queue_time_us":275,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":58,"raft.queue_time_us":13,"raft.run_cpu_time_us":2,"raft.run_wall_time_us":2,"replication_time_us":3991391}]]}
W20260809 19:21:06.372823 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:21:06.861729 4039 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKf1XAf/build/release/bin/kudu with pid 6004
W20260809 19:21:06.873672 6684 meta_cache.cc:302] tablet 7c092d80045840deac3151a94e01de0b: replica 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111) has failed: Network error: recv got EOF from 127.3.241.197:45111 (error 108)
I20260809 19:21:06.878151 6381 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 8 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 4 index: 1399. Preceding OpId from leader: term: 8 index: 2207. (index mismatch)
W20260809 19:21:06.878469 6270 connection.cc:441] server connection from 127.3.241.197:38703 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.197:38703 (request call id 19) could send its response
I20260809 19:21:06.878554 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 8 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 6 index: 2042. Preceding OpId from leader: term: 8 index: 2207. (index mismatch)
W20260809 19:21:06.878707 5805 connection.cc:441] server connection from 127.3.241.197:46963 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.197:46963 (request call id 95) could send its response
W20260809 19:21:06.879393 5377 connection.cc:441] server connection from 127.3.241.197:42739 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.197:42739 (request call id 227) could send its response
W20260809 19:21:06.880899 7905 connection.cc:441] server connection from 127.3.241.197:58637 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.197:58637 (request call id 142) could send its response
I20260809 19:21:06.881428 4039 test_util.cc:268] Using random seed: 598671244
I20260809 19:21:06.987895 4039 test_util.cc:268] Using random seed: 598777708
I20260809 19:21:07.192191 4039 test_util.cc:268] Using random seed: 598981969
I20260809 19:21:07.368944 8302 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:07.369057 8302 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:07.369366 8302 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:07.369596 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 9 candidate_status { last_received { term: 6 index: 2042 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:07.369689 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 9 candidate_status { last_received { term: 6 index: 2042 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:07.369676 7569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 9 candidate_status { last_received { term: 6 index: 2042 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:07.369675 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 9 candidate_status { last_received { term: 6 index: 2042 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:07.369928 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 9 candidate_status { last_received { term: 6 index: 2042 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:07.369974 5804 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:07.370038 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 8.
I20260809 19:21:07.370064 8302 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260809 19:21:07.370147 5805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260809 19:21:07.370543 5805 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:07.496445 4039 test_util.cc:268] Using random seed: 599286252
I20260809 19:21:07.900204 4039 test_util.cc:268] Using random seed: 599690016
I20260809 19:21:07.919138 8307 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:07.919271 8307 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:07.919595 8307 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:07.919848 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 9 candidate_status { last_received { term: 8 index: 2282 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:07.919855 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 9 candidate_status { last_received { term: 8 index: 2282 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:07.919919 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 9 candidate_status { last_received { term: 8 index: 2282 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:07.920004 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 8.
I20260809 19:21:07.919958 7569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 9 candidate_status { last_received { term: 8 index: 2282 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
W20260809 19:21:07.920189 5379 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:07.920264 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 9 candidate_status { last_received { term: 8 index: 2282 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:07.920372 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 8.
W20260809 19:21:07.920679 5379 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:07.920747 5379 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:07.920843 8307 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 19:21:08.033744 8311 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:08.033833 8312 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:08.033871 8311 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:08.033965 8312 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:08.034174 8311 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:08.034380 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:08.034477 8312 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
W20260809 19:21:08.034430 7465 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260809 19:21:08.034490 6586 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 8.
I20260809 19:21:08.034628 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:08.034749 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:08.034794 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:08.034796 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:08.034850 5479 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 8.
I20260809 19:21:08.034884 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 8.
I20260809 19:21:08.035017 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:08.035022 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:08.035110 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 8.
I20260809 19:21:08.035270 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:08.035356 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 8.
I20260809 19:21:08.035392 7569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:08.035436 7462 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468; no voters: 61427f2036ac4f96a9655aef910a7d7a
I20260809 19:21:08.035523 7569 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 8.
I20260809 19:21:08.035570 8311 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 19:21:08.035614 8311 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:08.035681 8311 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 8 FOLLOWER]: Advancing to term 9
I20260809 19:21:08.035728 6484 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters: 61427f2036ac4f96a9655aef910a7d7a, 7d4177500c7a4f41afce5190ff786506
I20260809 19:21:08.035743 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:08.035807 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 8.
W20260809 19:21:08.035846 6487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111) [suppressed 3 similar messages]
I20260809 19:21:08.035986 8312 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 19:21:08.036075 8312 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:08.036110 8312 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 8 FOLLOWER]: Advancing to term 9
W20260809 19:21:08.036334 7465 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
W20260809 19:21:08.036334 6487 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:08.036753 8311 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:08.037030 8312 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:08.037096 8311 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 9 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:08.037262 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:08.037371 6586 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 in current term 9: Already voted for candidate 23183da38752444a9ff5525be50cd404 in this term.
I20260809 19:21:08.037365 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:08.037418 6381 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 8 FOLLOWER]: Advancing to term 9
W20260809 19:21:08.037420 7465 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:08.037416 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:08.037497 5482 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 8 FOLLOWER]: Advancing to term 9
I20260809 19:21:08.037469 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:08.037536 5918 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 8 FOLLOWER]: Advancing to term 9
I20260809 19:21:08.037534 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:08.038167 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 9.
I20260809 19:21:08.038271 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 9.
I20260809 19:21:08.038245 8006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:08.038276 7569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:21:08.038350 7569 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 23183da38752444a9ff5525be50cd404 in current term 9: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:08.038446 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 9.
I20260809 19:21:08.038604 8312 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 9 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:08.038601 7461 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073
I20260809 19:21:08.038741 8311 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 19:21:08.038810 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:08.038890 5918 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 23183da38752444a9ff5525be50cd404 in current term 9: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:08.038902 8311 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 9 LEADER]: Becoming Leader. State: Replica: dd68273a4b314386ae0f5489e2f1d468, State: Running, Role: LEADER
W20260809 19:21:08.038929 6487 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:08.039036 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:08.039021 8311 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2498, Committed index: 2498, Last appended: 8.2506, Last appended by leader: 2506, Current term: 9, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:08.039096 5482 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 23183da38752444a9ff5525be50cd404 in current term 9: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:08.039171 6486 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 23183da38752444a9ff5525be50cd404; no voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:08.039220 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 9 candidate_status { last_received { term: 8 index: 2506 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:08.039294 6381 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 23183da38752444a9ff5525be50cd404 in current term 9: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:08.039319 8312 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260809 19:21:08.039855 5043 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 reported cstate change: term changed from 8 to 9, leader changed from 757ec8723e924451a89abb5cc62c1073 (127.3.241.197) to dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193). New cstate: current_term: 9 leader_uuid: "dd68273a4b314386ae0f5489e2f1d468" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
I20260809 19:21:08.383504 8333 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:08.383730 8333 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:08.384369 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 9 candidate_status { last_received { term: 8 index: 2499 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:08.384389 7569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 9 candidate_status { last_received { term: 8 index: 2499 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:08.384508 6586 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a in current term 9: Already voted for candidate 23183da38752444a9ff5525be50cd404 in this term.
I20260809 19:21:08.384701 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 9 candidate_status { last_received { term: 8 index: 2499 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:08.384806 5482 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a in current term 9: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:08.384812 8333 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:08.384814 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 9 candidate_status { last_received { term: 8 index: 2499 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:08.384927 5918 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a in current term 9: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:08.385085 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 9 candidate_status { last_received { term: 8 index: 2499 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:08.385214 6381 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a in current term 9: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
W20260809 19:21:08.385373 7904 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:08.385443 7904 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 1 yes votes; 5 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a; no voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:08.385577 8333 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 8 FOLLOWER]: Advancing to term 9
I20260809 19:21:08.386660 8333 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 19:21:08.404628 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.197:45111
--local_ip_for_outbound_sockets=127.3.241.197
--tserver_master_addrs=127.3.241.254:44125
--webserver_port=38231
--webserver_interface=127.3.241.197
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260809 19:21:08.426143 6381 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 9 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 4 index: 1399. Preceding OpId from leader: term: 9 index: 2507. (index mismatch)
I20260809 19:21:08.426455 8311 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2507, Last known committed idx: 1393, Time since last communication: 0.000s
I20260809 19:21:08.456322 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 9 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 6 index: 2042. Preceding OpId from leader: term: 9 index: 2507. (index mismatch)
I20260809 19:21:08.456815 8311 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2507, Last known committed idx: 2039, Time since last communication: 0.000s
I20260809 19:21:08.488633 5482 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 9 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 8 index: 2282. Preceding OpId from leader: term: 9 index: 2507. (index mismatch)
I20260809 19:21:08.491324 8311 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2507, Last known committed idx: 2279, Time since last communication: 0.000s
I20260809 19:21:08.496604 8006 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 9 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 8 index: 2499. Preceding OpId from leader: term: 9 index: 2507. (index mismatch)
I20260809 19:21:08.500415 8311 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2507, Last known committed idx: 2491, Time since last communication: 0.000s
I20260809 19:21:08.511981 6586 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 9 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 8 index: 2506. Preceding OpId from leader: term: 9 index: 2507. (index mismatch)
I20260809 19:21:08.513052 8311 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2507, Last known committed idx: 2498, Time since last communication: 0.000s
W20260809 19:21:08.513202 7465 consensus_peers.cc:487] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 -> Peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Couldn't send request to peer 757ec8723e924451a89abb5cc62c1073. Status: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 19:21:08.522182 8399 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2491, attempt_no=1}) took 1642 ms. Trace:
I20260809 19:21:08.522352 8399 rpcz_store.cc:276] 0809 19:21:06.879774 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:06.880395 (+ 621us) service_pool.cc:227] Handling call
0809 19:21:08.522158 (+1641763us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 19:21:08.523267 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:21:08.523590 8401 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2493, attempt_no=1}) took 1643 ms. Trace:
I20260809 19:21:08.523711 8401 rpcz_store.cc:276] 0809 19:21:06.879787 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:06.880283 (+ 496us) service_pool.cc:227] Handling call
0809 19:21:08.523572 (+1643289us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:21:08.523818 8402 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2494, attempt_no=1}) took 1644 ms. Trace:
I20260809 19:21:08.524029 8402 rpcz_store.cc:276] 0809 19:21:06.879708 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:06.880362 (+ 654us) service_pool.cc:227] Handling call
0809 19:21:08.523806 (+1643444us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:21:08.524246 8401 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2495, attempt_no=1}) took 1644 ms. Trace:
I20260809 19:21:08.524324 8401 rpcz_store.cc:276] 0809 19:21:06.879752 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:06.879861 (+ 109us) service_pool.cc:227] Handling call
0809 19:21:08.524242 (+1644381us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 19:21:08.524534 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:21:08.524683 8401 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2496, attempt_no=1}) took 1644 ms. Trace:
I20260809 19:21:08.524726 8401 rpcz_store.cc:276] 0809 19:21:06.879733 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:06.879820 (+ 87us) service_pool.cc:227] Handling call
0809 19:21:08.524681 (+1644861us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:21:08.524848 8401 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2497, attempt_no=1}) took 1645 ms. Trace:
I20260809 19:21:08.524885 8401 rpcz_store.cc:276] 0809 19:21:06.879692 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:06.880339 (+ 647us) service_pool.cc:227] Handling call
0809 19:21:08.524845 (+1644506us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 19:21:08.525554 5404 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.525749 5405 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
I20260809 19:21:08.525635 8401 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2498, attempt_no=1}) took 1645 ms. Trace:
I20260809 19:21:08.525802 8401 rpcz_store.cc:276] 0809 19:21:06.879799 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:06.879863 (+ 64us) service_pool.cc:227] Handling call
0809 19:21:08.525630 (+1645767us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1920}
W20260809 19:21:08.525554 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.526096 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.526825 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.526943 5405 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.527261 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:21:08.523021 8399 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=2492, attempt_no=1}) took 1643 ms. Trace:
I20260809 19:21:08.527742 8399 rpcz_store.cc:276] 0809 19:21:06.879722 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:06.880421 (+ 699us) service_pool.cc:227] Handling call
0809 19:21:08.523014 (+1642593us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 19:21:08.528357 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.529592 5405 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.529874 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.530422 5851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.530803 5851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.532318 5851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.533470 5851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.533486 5405 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.533607 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.533699 5850 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.533807 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.534334 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:08.537096 6310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:08.537190 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:08.538204 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:08.538260 6304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:08.538385 6310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:08.539297 6310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:08.540400 6310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:08.542637 6310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:08.546152 8337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:08.546446 8337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:08.546509 8337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:08.548130 8337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:08.548218 8337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.197
I20260809 19:21:08.551429 8337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.197:45111
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.3.241.197
--webserver_port=38231
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.8337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.197
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:08.551738 8337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:08.552139 8337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:08.564190 8417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:08.564532 8418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:08.564735 8421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:08.568110 8337 server_base.cc:1034] running on GCE node
I20260809 19:21:08.568434 8337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:08.568760 8337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:08.572059 8337 hybrid_clock.cc:648] HybridClock initialized: now 1786303268572029 us; error 38 us; skew 500 ppm
I20260809 19:21:08.576956 8337 webserver.cc:460] Webserver started at http://127.3.241.197:38231/ using document root <none> and password file <none>
I20260809 19:21:08.577297 8337 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:08.577384 8337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:08.580806 8337 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:08.583871 8439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:08.585320 8337 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260809 19:21:08.585451 8337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "757ec8723e924451a89abb5cc62c1073"
format_stamp: "Formatted at 2026-08-09 19:20:49 on dist-test-slave-gmjp"
I20260809 19:21:08.586097 8337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:08.617041 8337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:08.617462 8337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:08.617666 8337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:08.617986 8337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:08.618711 8465 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:21:08.631423 8337 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:21:08.631678 8337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.013s user 0.000s sys 0.000s
I20260809 19:21:08.631789 8337 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:21:08.634859 8337 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:21:08.634913 8337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260809 19:21:08.635846 8465 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Bootstrap starting.
I20260809 19:21:08.654053 8337 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.197:45111
I20260809 19:21:08.654755 8337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20260809 19:21:08.663805 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 8337
I20260809 19:21:08.673576 8653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.197:45111 every 8 connection(s)
I20260809 19:21:08.715364 8654 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:21:08.715512 8654 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:08.715770 8654 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:21:08.716682 5042 ts_manager.cc:194] Re-registered known tserver with Master: 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111)
I20260809 19:21:08.717307 5042 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.197:54799
I20260809 19:21:08.723332 8465 log.cc:824] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:09.002748 8465 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Bootstrap replayed 1/1 log segments. Stats: ops{read=2528 overwritten=22 applied=2498 ignored=0} inserts{seen=24910 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260809 19:21:09.003273 8465 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Bootstrap complete.
I20260809 19:21:09.004676 8465 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Time spent bootstrapping tablet: real 0.369s user 0.213s sys 0.115s
I20260809 19:21:09.006111 8465 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 8 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:09.007367 8465 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 757ec8723e924451a89abb5cc62c1073, State: Initialized, Role: FOLLOWER
I20260809 19:21:09.007567 8465 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2498, Last appended: 8.2506, Last appended by leader: 2506, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:09.007874 8654 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
I20260809 19:21:09.007870 8465 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 19:21:09.534528 4039 test_util.cc:268] Using random seed: 601324341
I20260809 19:21:09.839394 4039 test_util.cc:268] Using random seed: 601629207
I20260809 19:21:10.244217 4039 test_util.cc:268] Using random seed: 602034023
I20260809 19:21:10.472755 8674 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:21:10.472908 8675 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:21:10.472893 8674 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:10.473150 8674 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:10.473024 8675 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:10.473500 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:10.473507 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:10.473585 6586 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 9.
I20260809 19:21:10.473577 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:10.473584 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:10.473919 8675 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:10.474144 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:10.474156 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:10.474200 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:10.474236 8005 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 9.
I20260809 19:21:10.474359 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:10.477049 8574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:10.477167 8572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:10.477247 8574 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 8.
I20260809 19:21:10.477459 6487 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a; no voters: 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad
I20260809 19:21:10.477604 8675 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 19:21:10.749822 4039 test_util.cc:268] Using random seed: 602539636
I20260809 19:21:10.928225 8682 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:21:10.928356 8682 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:10.928645 8682 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:10.928849 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 10 candidate_status { last_received { term: 9 index: 2904 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:10.928972 6586 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for term 10 because replica has last-logged OpId of term: 9 index: 3057, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 2904.
I20260809 19:21:10.928977 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 10 candidate_status { last_received { term: 9 index: 2904 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:10.929021 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 10 candidate_status { last_received { term: 9 index: 2904 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:10.929102 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 9.
I20260809 19:21:10.929204 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 10 candidate_status { last_received { term: 9 index: 2904 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:10.929324 8005 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for term 10 because replica has last-logged OpId of term: 9 index: 3057, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 2904.
I20260809 19:21:10.929579 8687 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:21:10.929654 8687 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:10.930141 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 10 candidate_status { last_received { term: 9 index: 2767 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:10.930220 8005 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4c4b93755eb4aa48c04795ad08e09ad for term 10 because replica has last-logged OpId of term: 9 index: 3057, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 2767.
I20260809 19:21:10.930186 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 10 candidate_status { last_received { term: 9 index: 2767 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:10.930264 5482 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4c4b93755eb4aa48c04795ad08e09ad for term 10 because replica has last-logged OpId of term: 9 index: 3057, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 2767.
I20260809 19:21:10.930518 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 10 candidate_status { last_received { term: 9 index: 2767 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:10.930608 6586 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4c4b93755eb4aa48c04795ad08e09ad for term 10 because replica has last-logged OpId of term: 9 index: 3057, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 2767.
I20260809 19:21:10.930634 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 10 candidate_status { last_received { term: 9 index: 2767 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:10.930728 5918 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4c4b93755eb4aa48c04795ad08e09ad for term 10 because replica has last-logged OpId of term: 9 index: 2904, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 2767.
I20260809 19:21:10.930878 6270 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506
I20260809 19:21:10.931020 8689 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:21:10.931129 8690 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 19:21:10.931110 8689 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:10.931355 8689 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:10.931545 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:10.931617 8005 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 9.
I20260809 19:21:10.931666 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:10.931692 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:10.930125 8687 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545)
I20260809 19:21:10.931761 6586 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 9.
I20260809 19:21:10.931770 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 9.
I20260809 19:21:10.931854 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:10.931916 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 9.
I20260809 19:21:10.932216 5375 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters:
I20260809 19:21:10.933001 8572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 10 candidate_status { last_received { term: 9 index: 2904 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:10.933095 8572 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 8.
I20260809 19:21:10.932631 8689 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 19:21:10.933279 8689 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 9 FOLLOWER]: Starting leader election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:21:10.933341 8689 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 9 FOLLOWER]: Advancing to term 10
I20260809 19:21:10.934370 8689 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:10.934727 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:10.934804 6586 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 9 FOLLOWER]: Advancing to term 10
I20260809 19:21:10.934794 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:10.934872 8005 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 9 FOLLOWER]: Advancing to term 10
I20260809 19:21:10.934926 8572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 10 candidate_status { last_received { term: 9 index: 2767 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:10.935005 8572 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 8.
I20260809 19:21:10.935562 6586 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 10.
I20260809 19:21:10.935592 8005 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 10.
I20260809 19:21:10.935842 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:10.935932 5918 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 9 FOLLOWER]: Advancing to term 10
I20260809 19:21:10.935948 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:10.936028 6381 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 9 FOLLOWER]: Advancing to term 10
I20260809 19:21:10.936712 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 10.
I20260809 19:21:10.936795 8689 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 10 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:10.936919 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 10.
I20260809 19:21:10.936960 5377 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506; no voters:
I20260809 19:21:10.937114 8689 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 10 FOLLOWER]: Leader election won for term 10
I20260809 19:21:10.937198 8572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:10.937263 8689 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 10 LEADER]: Becoming Leader. State: Replica: 7d4177500c7a4f41afce5190ff786506, State: Running, Role: LEADER
I20260809 19:21:10.937233 8574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073"
I20260809 19:21:10.937276 8572 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 8.
I20260809 19:21:10.937381 8689 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3049, Committed index: 3049, Last appended: 9.3057, Last appended by leader: 3057, Current term: 10, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:10.938256 5042 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 reported cstate change: term changed from 9 to 10, leader changed from dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193) to 7d4177500c7a4f41afce5190ff786506 (127.3.241.194). New cstate: current_term: 10 leader_uuid: "7d4177500c7a4f41afce5190ff786506" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
I20260809 19:21:11.255599 7569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 10 candidate_status { last_received { term: 9 index: 2767 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:11.255746 7570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
W20260809 19:21:11.255977 7462 connection.cc:232] RPC call timeout handler was delayed by -1.4941073490772396s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260809 19:21:11.256109 7902 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073; no voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468
W20260809 19:21:11.256171 7461 connection.cc:232] RPC call timeout handler was delayed by -1.5964732931461185s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260809 19:21:11.256215 7461 connection.cc:232] RPC call timeout handler was delayed by -1.3840369791723788s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260809 19:21:11.256481 7556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 10 candidate_status { last_received { term: 9 index: 2904 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:11.256538 8704 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 19:21:11.256696 8005 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 10 FOLLOWER]: Rejecting Update request from peer dd68273a4b314386ae0f5489e2f1d468 for earlier term 9. Current term is 10. Ops: []
I20260809 19:21:11.257028 5803 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:11.257176 8682 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 19:21:11.257236 7555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:11.257414 7565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:21:11.257542 7566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 10 candidate_status { last_received { term: 9 index: 3057 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:11.257706 8460 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: INVALID_TERM, Last received: 9.3057, Next index: 3058, Last known committed idx: 3049, Time since last communication: 0.000s
I20260809 19:21:11.257838 8460 raft_consensus.cc:3032] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 9 LEADER]: Stepping down as leader of term 9
I20260809 19:21:11.257861 8460 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: dd68273a4b314386ae0f5489e2f1d468, State: Running, Role: LEADER
I20260809 19:21:11.257906 8460 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3049, Committed index: 3049, Last appended: 9.3057, Last appended by leader: 3057, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
W20260809 19:21:11.258168 8452 consensus_queue.cc:1159] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" responder_term: 9 status { last_received { term: 9 index: 2767 } last_committed_idx: 2766 last_received_current_leader { term: 9 index: 2767 } }
W20260809 19:21:11.258347 8456 consensus_queue.cc:1159] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "7d4177500c7a4f41afce5190ff786506" responder_term: 9 status { last_received { term: 9 index: 3057 } last_committed_idx: 3049 last_received_current_leader { term: 9 index: 3057 } }
I20260809 19:21:11.259121 5918 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 10 FOLLOWER]: Rejecting Update request from peer dd68273a4b314386ae0f5489e2f1d468 for earlier term 9. Current term is 10. Ops: [9.2905-9.3057]
I20260809 19:21:11.259255 6586 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 10 FOLLOWER]: Rejecting Update request from peer dd68273a4b314386ae0f5489e2f1d468 for earlier term 9. Current term is 10. Ops: []
I20260809 19:21:11.259814 8430 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: INVALID_TERM, Last received: 9.3057, Next index: 3058, Last known committed idx: 3049, Time since last communication: 0.000s
I20260809 19:21:11.260005 8460 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 9 FOLLOWER]: Advancing to term 10
I20260809 19:21:11.263199 8572 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 8 FOLLOWER]: Advancing to term 9
I20260809 19:21:11.281178 8712 mvcc.cc:205] Tried to move back new op lower bound from 7316698188838772736 to 7316698185888935936. Current Snapshot: MvccSnapshot[applied={T|T < 7316698188178468864 or (T in {7316698188178624512,7316698188179636224,7316698188180336640,7316698188185899008,7316698188194365440,7316698188194758656,7316698188195041280,7316698188195520512,7316698188195745792,7316698188195954688,7316698188220968960,7316698188221485056,7316698188221894656,7316698188223275008,7316698188223479808,7316698188223655936,7316698188223815680,7316698188234665984})}]
I20260809 19:21:11.367920 8572 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 9 FOLLOWER]: Advancing to term 10
I20260809 19:21:11.368772 8572 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 10 FOLLOWER]: Refusing update from remote peer 7d4177500c7a4f41afce5190ff786506: Log matching property violated. Preceding OpId in replica: term: 9 index: 3057. Preceding OpId from leader: term: 10 index: 3058. (index mismatch)
I20260809 19:21:11.369115 8689 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3058, Last known committed idx: 3049, Time since last communication: 0.000s
I20260809 19:21:11.452183 8005 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 10 FOLLOWER]: Refusing update from remote peer 7d4177500c7a4f41afce5190ff786506: Log matching property violated. Preceding OpId in replica: term: 9 index: 3057. Preceding OpId from leader: term: 10 index: 3058. (index mismatch)
I20260809 19:21:11.452483 8689 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3058, Last known committed idx: 3049, Time since last communication: 0.000s
I20260809 19:21:11.533237 7570 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 10 FOLLOWER]: Refusing update from remote peer 7d4177500c7a4f41afce5190ff786506: Log matching property violated. Preceding OpId in replica: term: 9 index: 3057. Preceding OpId from leader: term: 10 index: 3058. (index mismatch)
I20260809 19:21:11.533555 8689 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3058, Last known committed idx: 3049, Time since last communication: 0.000s
I20260809 19:21:11.535367 6586 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 10 FOLLOWER]: Refusing update from remote peer 7d4177500c7a4f41afce5190ff786506: Log matching property violated. Preceding OpId in replica: term: 9 index: 3057. Preceding OpId from leader: term: 10 index: 3058. (index mismatch)
I20260809 19:21:11.535632 8719 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3058, Last known committed idx: 3049, Time since last communication: 0.000s
I20260809 19:21:11.535758 8734 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43162 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=3041, attempt_no=0}) took 2566 ms. Trace:
I20260809 19:21:11.535837 8734 rpcz_store.cc:276] 0809 19:21:08.969142 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:08.969163 (+ 21us) service_pool.cc:227] Handling call
0809 19:21:11.535747 (+2566584us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:08.969262 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:08.969280 (+ 18us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:08.969281 (+ 1us) write_op.cc:435] Acquired schema lock
0809 19:21:08.969281 (+ 0us) tablet.cc:660] Decoding operations
0809 19:21:08.969294 (+ 13us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:08.969295 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:08.969296 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:08.969307 (+ 11us) tablet.cc:699] Row locks acquired
0809 19:21:08.969308 (+ 1us) write_op.cc:260] PREPARE: finished
0809 19:21:08.969311 (+ 3us) write_op.cc:270] Start()
0809 19:21:08.969313 (+ 2us) write_op.cc:276] Timestamp: P: 1786303268969302 usec, L: 0
0809 19:21:08.969314 (+ 1us) op_driver.cc:331] REPLICATION: starting
0809 19:21:08.969336 (+ 22us) log.cc:842] Serialized 686 byte log entry
0809 19:21:11.534835 (+2565499us) op_driver.cc:444] REPLICATION: finished
0809 19:21:11.535642 (+ 807us) write_op.cc:301] APPLY: starting
0809 19:21:11.535645 (+ 3us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:11.535652 (+ 7us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:11.535665 (+ 13us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:11.535688 (+ 23us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:11.535692 (+ 4us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:11.535693 (+ 1us) write_op.cc:312] APPLY: finished
0809 19:21:11.535708 (+ 15us) log.cc:842] Serialized 77 byte log entry
0809 19:21:11.535721 (+ 13us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:11.535726 (+ 5us) write_op.cc:454] Released schema lock
0809 19:21:11.535726 (+ 0us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":795,"num_ops":10,"prepare.queue_time_us":40,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":105,"raft.queue_time_us":38,"raft.run_cpu_time_us":22,"raft.run_wall_time_us":23,"replication_time_us":2565517,"thread_start_us":58,"threads_started":1}]]}
I20260809 19:21:11.537333 8734 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43162 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=3042, attempt_no=0}) took 2568 ms. Trace:
I20260809 19:21:11.537388 8734 rpcz_store.cc:276] 0809 19:21:08.969244 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:08.969385 (+ 141us) service_pool.cc:227] Handling call
0809 19:21:11.537328 (+2567943us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:08.969578 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:08.969584 (+ 6us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:08.969584 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:21:08.969584 (+ 0us) tablet.cc:660] Decoding operations
0809 19:21:08.969594 (+ 10us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:08.969596 (+ 2us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:08.969596 (+ 0us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:08.969606 (+ 10us) tablet.cc:699] Row locks acquired
0809 19:21:08.969606 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:21:08.969609 (+ 3us) write_op.cc:270] Start()
0809 19:21:08.969611 (+ 2us) write_op.cc:276] Timestamp: P: 1786303268969599 usec, L: 0
0809 19:21:08.969611 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:21:08.969629 (+ 18us) log.cc:842] Serialized 696 byte log entry
0809 19:21:11.534904 (+2565275us) op_driver.cc:444] REPLICATION: finished
0809 19:21:11.537240 (+ 2336us) write_op.cc:301] APPLY: starting
0809 19:21:11.537243 (+ 3us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:11.537248 (+ 5us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:11.537249 (+ 1us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:11.537283 (+ 34us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:11.537288 (+ 5us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:11.537288 (+ 0us) write_op.cc:312] APPLY: finished
0809 19:21:11.537302 (+ 14us) log.cc:842] Serialized 77 byte log entry
0809 19:21:11.537308 (+ 6us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:11.537313 (+ 5us) write_op.cc:454] Released schema lock
0809 19:21:11.537314 (+ 1us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2330,"num_ops":10,"prepare.queue_time_us":134,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":70,"raft.queue_time_us":20,"raft.run_cpu_time_us":2,"raft.run_wall_time_us":2,"replication_time_us":2565291,"thread_start_us":53,"threads_started":1}]]}
I20260809 19:21:11.537599 8734 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43162 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=3043, attempt_no=0}) took 2568 ms. Trace:
W20260809 19:21:11.537767 7492 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43162: Illegal state: replica dd68273a4b314386ae0f5489e2f1d468 is not leader of this config: current role FOLLOWER
I20260809 19:21:11.537918 8732 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43162 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=3046, attempt_no=0}) took 2568 ms. Trace:
I20260809 19:21:11.537753 8734 rpcz_store.cc:276] 0809 19:21:08.969480 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:08.969506 (+ 26us) service_pool.cc:227] Handling call
0809 19:21:11.537596 (+2568090us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:08.969652 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:08.969657 (+ 5us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:08.969657 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:21:08.969657 (+ 0us) tablet.cc:660] Decoding operations
0809 19:21:08.969665 (+ 8us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:08.969666 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:08.969667 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:08.969674 (+ 7us) tablet.cc:699] Row locks acquired
0809 19:21:08.969674 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:21:08.969676 (+ 2us) write_op.cc:270] Start()
0809 19:21:08.969677 (+ 1us) write_op.cc:276] Timestamp: P: 1786303268969667 usec, L: 0
0809 19:21:08.969678 (+ 1us) op_driver.cc:331] REPLICATION: starting
0809 19:21:08.969687 (+ 9us) log.cc:842] Serialized 696 byte log entry
0809 19:21:11.534963 (+2565276us) op_driver.cc:444] REPLICATION: finished
0809 19:21:11.537538 (+ 2575us) write_op.cc:301] APPLY: starting
0809 19:21:11.537540 (+ 2us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:11.537544 (+ 4us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:11.537545 (+ 1us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:11.537562 (+ 17us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:11.537565 (+ 3us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:11.537565 (+ 0us) write_op.cc:312] APPLY: finished
0809 19:21:11.537577 (+ 12us) log.cc:842] Serialized 77 byte log entry
0809 19:21:11.537580 (+ 3us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:11.537585 (+ 5us) write_op.cc:454] Released schema lock
0809 19:21:11.537585 (+ 0us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2568,"num_ops":10,"prepare.queue_time_us":89,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":17,"raft.run_cpu_time_us":1,"raft.run_wall_time_us":2,"replication_time_us":2565284,"thread_start_us":72,"threads_started":1}]]}
W20260809 19:21:11.538019 7493 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43162: Illegal state: replica dd68273a4b314386ae0f5489e2f1d468 is not leader of this config: current role FOLLOWER
I20260809 19:21:11.538044 8732 rpcz_store.cc:276] 0809 19:21:08.969527 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:08.969573 (+ 46us) service_pool.cc:227] Handling call
0809 19:21:11.537913 (+2568340us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:08.969704 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:08.969707 (+ 3us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:08.969708 (+ 1us) write_op.cc:435] Acquired schema lock
0809 19:21:08.969708 (+ 0us) tablet.cc:660] Decoding operations
0809 19:21:08.969717 (+ 9us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:08.969717 (+ 0us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:08.969718 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:08.969724 (+ 6us) tablet.cc:699] Row locks acquired
0809 19:21:08.969724 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:21:08.969726 (+ 2us) write_op.cc:270] Start()
0809 19:21:08.969727 (+ 1us) write_op.cc:276] Timestamp: P: 1786303268969716 usec, L: 0
0809 19:21:08.969727 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:21:08.969735 (+ 8us) log.cc:842] Serialized 696 byte log entry
0809 19:21:11.535042 (+2565307us) op_driver.cc:444] REPLICATION: finished
0809 19:21:11.537821 (+ 2779us) write_op.cc:301] APPLY: starting
0809 19:21:11.537823 (+ 2us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:11.537827 (+ 4us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:11.537841 (+ 14us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:11.537863 (+ 22us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:11.537866 (+ 3us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:11.537867 (+ 1us) write_op.cc:312] APPLY: finished
0809 19:21:11.537880 (+ 13us) log.cc:842] Serialized 77 byte log entry
0809 19:21:11.537893 (+ 13us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:11.537899 (+ 6us) write_op.cc:454] Released schema lock
0809 19:21:11.537899 (+ 0us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2771,"num_ops":10,"prepare.queue_time_us":106,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":17,"raft.run_cpu_time_us":1,"raft.run_wall_time_us":2,"replication_time_us":2565312,"thread_start_us":53,"threads_started":1}]]}
I20260809 19:21:11.538348 8730 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43162 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=3047, attempt_no=0}) took 2568 ms. Trace:
I20260809 19:21:11.538445 8734 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43162 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=3044, attempt_no=0}) took 2568 ms. Trace:
W20260809 19:21:11.538669 7493 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43162: Illegal state: replica dd68273a4b314386ae0f5489e2f1d468 is not leader of this config: current role FOLLOWER
I20260809 19:21:11.538800 8734 rpcz_store.cc:276] 0809 19:21:08.969499 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:08.969573 (+ 74us) service_pool.cc:227] Handling call
0809 19:21:11.538438 (+2568865us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:08.969798 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:08.969802 (+ 4us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:08.969802 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:21:08.969802 (+ 0us) tablet.cc:660] Decoding operations
0809 19:21:08.969811 (+ 9us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:08.969811 (+ 0us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:08.969812 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:08.969818 (+ 6us) tablet.cc:699] Row locks acquired
0809 19:21:08.969818 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:21:08.969820 (+ 2us) write_op.cc:270] Start()
0809 19:21:08.969821 (+ 1us) write_op.cc:276] Timestamp: P: 1786303268969810 usec, L: 0
0809 19:21:08.969821 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:21:08.969829 (+ 8us) log.cc:842] Serialized 696 byte log entry
0809 19:21:11.535107 (+2565278us) op_driver.cc:444] REPLICATION: finished
0809 19:21:11.538287 (+ 3180us) write_op.cc:301] APPLY: starting
0809 19:21:11.538289 (+ 2us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:11.538294 (+ 5us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:11.538296 (+ 2us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:11.538328 (+ 32us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:11.538332 (+ 4us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:11.538332 (+ 0us) write_op.cc:312] APPLY: finished
0809 19:21:11.538346 (+ 14us) log.cc:842] Serialized 77 byte log entry
0809 19:21:11.538416 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:11.538422 (+ 6us) write_op.cc:454] Released schema lock
0809 19:21:11.538422 (+ 0us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3174,"num_ops":10,"prepare.queue_time_us":169,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":17,"raft.run_cpu_time_us":2,"raft.run_wall_time_us":2,"replication_time_us":2565284}]]}
I20260809 19:21:11.538743 8730 rpcz_store.cc:276] 0809 19:21:08.969542 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:08.969603 (+ 61us) service_pool.cc:227] Handling call
0809 19:21:11.538341 (+2568738us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:08.969751 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:08.969755 (+ 4us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:08.969755 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:21:08.969755 (+ 0us) tablet.cc:660] Decoding operations
0809 19:21:08.969764 (+ 9us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:08.969764 (+ 0us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:08.969765 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:08.969770 (+ 5us) tablet.cc:699] Row locks acquired
0809 19:21:08.969770 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:21:08.969772 (+ 2us) write_op.cc:270] Start()
0809 19:21:08.969774 (+ 2us) write_op.cc:276] Timestamp: P: 1786303268969763 usec, L: 0
0809 19:21:08.969774 (+ 0us) op_driver.cc:331] REPLICATION: starting
0809 19:21:08.969782 (+ 8us) log.cc:842] Serialized 696 byte log entry
0809 19:21:11.535104 (+2565322us) op_driver.cc:444] REPLICATION: finished
0809 19:21:11.538217 (+ 3113us) write_op.cc:301] APPLY: starting
0809 19:21:11.538219 (+ 2us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:11.538231 (+ 12us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:11.538235 (+ 4us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:11.538252 (+ 17us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:11.538255 (+ 3us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:11.538255 (+ 0us) write_op.cc:312] APPLY: finished
0809 19:21:11.538268 (+ 13us) log.cc:842] Serialized 77 byte log entry
0809 19:21:11.538324 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:11.538328 (+ 4us) write_op.cc:454] Released schema lock
0809 19:21:11.538329 (+ 1us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3105,"num_ops":10,"prepare.queue_time_us":129,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":18,"raft.run_cpu_time_us":2,"raft.run_wall_time_us":1,"replication_time_us":2565326}]]}
I20260809 19:21:11.539067 8734 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43162 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=3048, attempt_no=0}) took 2569 ms. Trace:
I20260809 19:21:11.539301 8734 rpcz_store.cc:276] 0809 19:21:08.969555 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:08.969908 (+ 353us) service_pool.cc:227] Handling call
0809 19:21:11.539064 (+2569156us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:08.970013 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:08.970019 (+ 6us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:08.970019 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:21:08.970019 (+ 0us) tablet.cc:660] Decoding operations
0809 19:21:08.970039 (+ 20us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:08.970040 (+ 1us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:08.970040 (+ 0us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:08.970048 (+ 8us) tablet.cc:699] Row locks acquired
0809 19:21:08.970048 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:21:08.970052 (+ 4us) write_op.cc:270] Start()
0809 19:21:08.970053 (+ 1us) write_op.cc:276] Timestamp: P: 1786303268970042 usec, L: 0
0809 19:21:08.970054 (+ 1us) op_driver.cc:331] REPLICATION: starting
0809 19:21:08.970076 (+ 22us) log.cc:842] Serialized 686 byte log entry
0809 19:21:11.535114 (+2565038us) op_driver.cc:444] REPLICATION: finished
0809 19:21:11.538986 (+ 3872us) write_op.cc:301] APPLY: starting
0809 19:21:11.538988 (+ 2us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:11.538992 (+ 4us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:11.538994 (+ 2us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:11.539012 (+ 18us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:11.539016 (+ 4us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:11.539017 (+ 1us) write_op.cc:312] APPLY: finished
0809 19:21:11.539028 (+ 11us) log.cc:842] Serialized 77 byte log entry
0809 19:21:11.539046 (+ 18us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:11.539051 (+ 5us) write_op.cc:454] Released schema lock
0809 19:21:11.539051 (+ 0us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3867,"num_ops":10,"prepare.queue_time_us":47,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":104,"raft.queue_time_us":19,"raft.run_cpu_time_us":2,"raft.run_wall_time_us":2,"replication_time_us":2565058}]]}
W20260809 19:21:11.539454 7493 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43162: Illegal state: replica dd68273a4b314386ae0f5489e2f1d468 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.539618 7492 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43162: Illegal state: replica dd68273a4b314386ae0f5489e2f1d468 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.539820 7502 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43162: Illegal state: replica dd68273a4b314386ae0f5489e2f1d468 is not leader of this config: current role FOLLOWER
I20260809 19:21:11.539306 8726 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43162 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=3045, attempt_no=0}) took 2569 ms. Trace:
I20260809 19:21:11.539928 8726 rpcz_store.cc:276] 0809 19:21:08.969515 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:08.969641 (+ 126us) service_pool.cc:227] Handling call
0809 19:21:11.539301 (+2569660us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 19:21:08.969846 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 7c092d80045840deac3151a94e01de0b
0809 19:21:08.969850 (+ 4us) write_op.cc:432] Acquiring schema lock in shared mode
0809 19:21:08.969850 (+ 0us) write_op.cc:435] Acquired schema lock
0809 19:21:08.969851 (+ 1us) tablet.cc:660] Decoding operations
0809 19:21:08.969859 (+ 8us) write_op.cc:620] Acquiring the partition lock for write op
0809 19:21:08.969859 (+ 0us) write_op.cc:641] Partition lock acquired for write op
0809 19:21:08.969860 (+ 1us) tablet.cc:683] Acquiring locks for 10 operations
0809 19:21:08.969866 (+ 6us) tablet.cc:699] Row locks acquired
0809 19:21:08.969866 (+ 0us) write_op.cc:260] PREPARE: finished
0809 19:21:08.969867 (+ 1us) write_op.cc:270] Start()
0809 19:21:08.969868 (+ 1us) write_op.cc:276] Timestamp: P: 1786303268969858 usec, L: 0
0809 19:21:08.969869 (+ 1us) op_driver.cc:331] REPLICATION: starting
0809 19:21:08.969876 (+ 7us) log.cc:842] Serialized 696 byte log entry
0809 19:21:11.535111 (+2565235us) op_driver.cc:444] REPLICATION: finished
0809 19:21:11.538834 (+ 3723us) write_op.cc:301] APPLY: starting
0809 19:21:11.538836 (+ 2us) tablet.cc:1365] starting BulkCheckPresence
0809 19:21:11.538841 (+ 5us) tablet.cc:1368] finished BulkCheckPresence
0809 19:21:11.538844 (+ 3us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 19:21:11.538866 (+ 22us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 19:21:11.538869 (+ 3us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0809 19:21:11.538869 (+ 0us) write_op.cc:312] APPLY: finished
0809 19:21:11.538881 (+ 12us) log.cc:842] Serialized 77 byte log entry
0809 19:21:11.539284 (+ 403us) write_op.cc:489] Releasing partition, row and schema locks
0809 19:21:11.539288 (+ 4us) write_op.cc:454] Released schema lock
0809 19:21:11.539288 (+ 0us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3714,"num_ops":10,"prepare.queue_time_us":177,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":17,"raft.run_cpu_time_us":1,"raft.run_wall_time_us":2,"replication_time_us":2565239}]]}
W20260809 19:21:11.540374 7492 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43162: Illegal state: replica dd68273a4b314386ae0f5489e2f1d468 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.540513 7502 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43162: Illegal state: replica dd68273a4b314386ae0f5489e2f1d468 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.545575 8499 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.545576 8493 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.545575 8500 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.545576 8492 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.545867 8498 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.546057 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.546167 8496 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.546183 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.548868 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.548863 6531 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.548863 6527 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.548868 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.549844 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.550951 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.552084 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:11.553233 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:21:11.947324 6381 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 10 FOLLOWER]: Refusing update from remote peer 7d4177500c7a4f41afce5190ff786506: Log matching property violated. Preceding OpId in replica: term: 9 index: 2767. Preceding OpId from leader: term: 10 index: 3058. (index mismatch)
I20260809 19:21:11.948733 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 10 FOLLOWER]: Refusing update from remote peer 7d4177500c7a4f41afce5190ff786506: Log matching property violated. Preceding OpId in replica: term: 9 index: 2904. Preceding OpId from leader: term: 10 index: 3058. (index mismatch)
W20260809 19:21:11.955317 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:11.955364 7942 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:11.955328 7944 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:11.955461 7948 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:11.955381 7949 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:11.955317 7951 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:11.955327 7941 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:11.955404 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
I20260809 19:21:12.054162 4039 test_util.cc:268] Using random seed: 603843974
I20260809 19:21:12.359269 4039 test_util.cc:268] Using random seed: 604149082
I20260809 19:21:12.764322 4039 test_util.cc:268] Using random seed: 604554134
I20260809 19:21:13.035292 8756 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d4177500c7a4f41afce5190ff786506)
I20260809 19:21:13.035429 8756 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:13.035739 8756 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:13.035972 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:13.036016 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:13.036028 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:13.036108 6586 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 10.
I20260809 19:21:13.036118 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 10.
I20260809 19:21:13.036118 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 10.
I20260809 19:21:13.036096 8572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:13.036213 8572 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 10.
I20260809 19:21:13.036319 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:13.036410 7462 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters:
I20260809 19:21:13.036530 8756 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260809 19:21:13.036484 8757 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 7d4177500c7a4f41afce5190ff786506)
I20260809 19:21:13.036603 8756 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 7d4177500c7a4f41afce5190ff786506)
I20260809 19:21:13.036630 8756 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 10 FOLLOWER]: Advancing to term 11
I20260809 19:21:13.036614 8757 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:13.037058 8757 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:13.037906 8756 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:13.038261 8572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073"
I20260809 19:21:13.038344 8572 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 10 FOLLOWER]: Advancing to term 11
I20260809 19:21:13.039076 8572 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 11.
I20260809 19:21:13.039445 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:13.039534 6586 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 10 FOLLOWER]: Advancing to term 11
I20260809 19:21:13.040330 6586 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 11.
I20260809 19:21:13.040671 8756 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 11 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:13.040730 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:13.040860 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:13.040947 5918 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 10 FOLLOWER]: Advancing to term 11
I20260809 19:21:13.041710 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 11.
I20260809 19:21:13.041832 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:13.041917 6381 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 10 FOLLOWER]: Advancing to term 11
I20260809 19:21:13.042662 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 11.
I20260809 19:21:13.042723 7462 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, dd68273a4b314386ae0f5489e2f1d468; no voters: 61427f2036ac4f96a9655aef910a7d7a
I20260809 19:21:13.043015 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:13.043089 6381 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 757ec8723e924451a89abb5cc62c1073 in current term 11: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:13.043107 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:13.043181 5918 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 757ec8723e924451a89abb5cc62c1073 in current term 11: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:13.043452 8756 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 11 FOLLOWER]: Leader election won for term 11
I20260809 19:21:13.043617 8756 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 11 LEADER]: Becoming Leader. State: Replica: dd68273a4b314386ae0f5489e2f1d468, State: Running, Role: LEADER
I20260809 19:21:13.043723 8756 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3058, Committed index: 3058, Last appended: 10.3058, Last appended by leader: 3058, Current term: 11, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:13.043780 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:13.043852 6586 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 757ec8723e924451a89abb5cc62c1073 in current term 11: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:13.044523 5040 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 reported cstate change: term changed from 10 to 11, leader changed from 7d4177500c7a4f41afce5190ff786506 (127.3.241.194) to dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193). New cstate: current_term: 11 leader_uuid: "dd68273a4b314386ae0f5489e2f1d468" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
I20260809 19:21:13.044946 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:13.045014 7570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:13.045181 8440 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 757ec8723e924451a89abb5cc62c1073; no voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:13.045363 8757 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260809 19:21:13.170317 8689 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3058, Last known committed idx: 2899, Time since last communication: 0.000s
I20260809 19:21:13.170603 5472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:13.170668 8719 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3058, Last known committed idx: 2766, Time since last communication: 0.000s
I20260809 19:21:13.170753 5473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:13.171931 6586 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 11 FOLLOWER]: Rejecting Update request from peer 7d4177500c7a4f41afce5190ff786506 for earlier term 10. Current term is 11. Ops: []
I20260809 19:21:13.172410 8719 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: INVALID_TERM, Last received: 10.3058, Next index: 3059, Last known committed idx: 3058, Time since last communication: 0.000s
I20260809 19:21:13.172441 8572 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 11 FOLLOWER]: Rejecting Update request from peer 7d4177500c7a4f41afce5190ff786506 for earlier term 10. Current term is 11. Ops: []
I20260809 19:21:13.172514 8719 raft_consensus.cc:3032] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 10 LEADER]: Stepping down as leader of term 10
I20260809 19:21:13.172541 8719 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 7d4177500c7a4f41afce5190ff786506, State: Running, Role: LEADER
I20260809 19:21:13.172585 8719 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3058, Committed index: 3058, Last appended: 10.3066, Last appended by leader: 3066, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:13.172706 8719 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 10 FOLLOWER]: Advancing to term 11
I20260809 19:21:13.174371 7570 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 11 LEADER]: Rejecting Update request from peer 7d4177500c7a4f41afce5190ff786506 for earlier term 10. Current term is 11. Ops: []
I20260809 19:21:13.174486 5918 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 11 FOLLOWER]: Rejecting Update request from peer 7d4177500c7a4f41afce5190ff786506 for earlier term 10. Current term is 11. Ops: [9.2905-10.3058]
I20260809 19:21:13.175038 6381 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 11 FOLLOWER]: Rejecting Update request from peer 7d4177500c7a4f41afce5190ff786506 for earlier term 10. Current term is 11. Ops: [9.2768-10.3058]
I20260809 19:21:13.176614 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 11 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:13.176709 5482 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 757ec8723e924451a89abb5cc62c1073 for term 11 because replica has last-logged OpId of term: 10 index: 3066, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 3058.
I20260809 19:21:13.455818 6586 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 11 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 10 index: 3058. Preceding OpId from leader: term: 11 index: 3059. (index mismatch)
I20260809 19:21:13.456202 8773 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3059, Last known committed idx: 3058, Time since last communication: 0.000s
I20260809 19:21:13.512928 8005 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 10 FOLLOWER]: Advancing to term 11
I20260809 19:21:13.513712 8005 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 11 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 10 index: 3061. Preceding OpId from leader: term: 11 index: 3059. (term mismatch)
I20260809 19:21:13.513765 8005 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Aborting all ops after (but not including) 3058
I20260809 19:21:13.513780 8005 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Aborting uncommitted WRITE_OP operation due to leader change: 10.3059
I20260809 19:21:13.513883 8005 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Aborting uncommitted WRITE_OP operation due to leader change: 10.3060
I20260809 19:21:13.513911 8005 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Aborting uncommitted WRITE_OP operation due to leader change: 10.3061
I20260809 19:21:13.514219 8773 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3059, Last known committed idx: 3058, Time since last communication: 0.000s
I20260809 19:21:13.646347 8572 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 11 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 10 index: 3058. Preceding OpId from leader: term: 11 index: 3059. (index mismatch)
I20260809 19:21:13.646674 8773 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3059, Last known committed idx: 3058, Time since last communication: 0.000s
I20260809 19:21:13.859922 4039 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKf1XAf/build/release/bin/kudu with pid 7426
I20260809 19:21:13.873275 6381 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 11 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 9 index: 2767. Preceding OpId from leader: term: 11 index: 3059. (index mismatch)
I20260809 19:21:13.874107 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 11 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 9 index: 2904. Preceding OpId from leader: term: 11 index: 3059. (index mismatch)
I20260809 19:21:13.874274 5482 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 11 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 10 index: 3066. Preceding OpId from leader: term: 11 index: 3059. (term mismatch)
I20260809 19:21:13.874328 5482 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: Aborting all ops after (but not including) 3058
I20260809 19:21:13.874347 5482 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: Aborting uncommitted WRITE_OP operation due to leader change: 10.3059
W20260809 19:21:13.874465 5482 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Aborted: Op aborted by new leader
I20260809 19:21:13.874568 5482 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: Aborting uncommitted WRITE_OP operation due to leader change: 10.3060
W20260809 19:21:13.874620 5482 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Aborted: Op aborted by new leader
I20260809 19:21:13.874673 5482 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: Aborting uncommitted WRITE_OP operation due to leader change: 10.3061
W20260809 19:21:13.874708 5482 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Aborted: Op aborted by new leader
I20260809 19:21:13.874749 5482 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: Aborting uncommitted WRITE_OP operation due to leader change: 10.3062
W20260809 19:21:13.874781 5482 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Aborted: Op aborted by new leader
I20260809 19:21:13.874823 5482 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: Aborting uncommitted WRITE_OP operation due to leader change: 10.3063
W20260809 19:21:13.874863 5482 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Aborted: Op aborted by new leader
I20260809 19:21:13.874905 5482 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: Aborting uncommitted WRITE_OP operation due to leader change: 10.3064
W20260809 19:21:13.874961 5482 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Aborted: Op aborted by new leader
I20260809 19:21:13.875000 5482 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: Aborting uncommitted WRITE_OP operation due to leader change: 10.3065
W20260809 19:21:13.875027 5482 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Aborted: Op aborted by new leader
I20260809 19:21:13.875092 5482 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506: Aborting uncommitted WRITE_OP operation due to leader change: 10.3066
W20260809 19:21:13.875144 5482 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Aborted: Op aborted by new leader
W20260809 19:21:13.876760 5806 connection.cc:441] server connection from 127.3.241.193:33427 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.193:33427 (request call id 125) could send its response
W20260809 19:21:13.876775 5379 connection.cc:441] server connection from 127.3.241.193:39917 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.193:39917 (request call id 251) could send its response
W20260809 19:21:13.876870 6270 connection.cc:441] server connection from 127.3.241.193:44115 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.193:44115 (request call id 141) could send its response
W20260809 19:21:13.876878 7904 connection.cc:441] server connection from 127.3.241.193:45441 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.193:45441 (request call id 226) could send its response
W20260809 19:21:13.876983 6488 connection.cc:441] server connection from 127.3.241.193:52713 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.193:52713 (request call id 188) could send its response
I20260809 19:21:13.878942 4039 test_util.cc:268] Using random seed: 605668758
W20260809 19:21:13.882407 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.883493 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.883533 5850 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.884629 5850 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.885794 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.885766 5850 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.885771 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.886884 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.891688 6310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:13.893836 6310 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:13.893893 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:13.893898 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:13.893898 6304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:13.894985 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:13.896031 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:13.896104 6304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:13.900372 6681 meta_cache.cc:302] tablet 7c092d80045840deac3151a94e01de0b: replica dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757) has failed: Network error: Client connection negotiation failed: client connection to 127.3.241.193:46757: connect: Connection refused (error 111)
W20260809 19:21:13.912467 8496 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.914469 8496 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.914496 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.916625 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.917730 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.917760 8496 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.918853 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.922065 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.923686 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.924711 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.926918 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.928061 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.928103 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.929148 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.930346 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.935607 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.937381 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:13.938407 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:13.939461 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:13.939586 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:13.941573 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:13.942628 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:13.943761 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:13.950081 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:13.951771 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.952260 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.952337 5411 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.953814 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.953824 5411 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.955397 5411 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.955425 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.962117 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.964968 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.966025 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.968298 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.968362 5850 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.968362 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.969444 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.969455 5850 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.977780 5850 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:13.979574 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:13.979565 6304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
I20260809 19:21:13.982681 4039 test_util.cc:268] Using random seed: 605772497
W20260809 19:21:13.982745 6304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:13.983843 6304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:13.983872 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:13.983999 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:13.984987 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:13.994345 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.018432 8496 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.018378 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.018378 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.020421 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.021557 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.032011 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.035920 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.036934 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.036988 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.037978 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.039139 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.039166 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.042166 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.050478 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.056411 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.058493 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.058594 7941 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.058594 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.059568 7941 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.060724 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.060724 7941 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.073212 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.076122 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.077843 5411 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.077844 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.078310 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.080966 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.080966 5411 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.083716 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.094395 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.096103 5850 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.101356 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.101323 5850 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.101348 5851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.101348 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.102385 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.107642 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.119143 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.119282 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.122910 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.125284 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.125481 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.126420 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.126483 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.129633 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.143918 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.148277 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.152640 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.172072 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.172075 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.173830 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.174930 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
I20260809 19:21:14.186698 4039 test_util.cc:268] Using random seed: 605976513
W20260809 19:21:14.190317 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.191460 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.196285 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.199452 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.199512 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.199664 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.200403 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.200793 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.216264 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.220093 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.225165 7941 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.225198 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.225198 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.226043 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.226043 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.227113 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.245658 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.249703 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.251322 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.252889 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.252889 5411 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.254568 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.254536 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.254536 5411 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.275372 5411 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.278286 5411 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.280016 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.281071 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.282158 5851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.282158 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.284408 5851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.284446 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.306789 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.306839 5851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.311116 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.312863 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.312867 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.313864 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.313890 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.314965 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.335538 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.338543 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.342833 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.344800 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.376371 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.376452 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.379343 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.379351 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.400774 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.407757 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.408761 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.411091 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.412230 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.412904 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.414040 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:21:14.428849 8824 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:14.428973 8824 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:14.429252 8824 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:14.429451 8572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 12 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:14.429574 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 12 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:14.429692 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 11.
I20260809 19:21:14.429766 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 12 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:14.429827 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 12 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:14.429883 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 12 candidate_status { last_received { term: 10 index: 3058 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:14.429934 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 11.
W20260809 19:21:14.430097 5375 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.193:46757: connect: Connection refused (error 111)
W20260809 19:21:14.430588 5375 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757): Network error: Client connection negotiation failed: client connection to 127.3.241.193:46757: connect: Connection refused (error 111)
I20260809 19:21:14.430636 5375 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:14.430809 8824 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260809 19:21:14.436587 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.439644 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.443524 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.444458 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.445569 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.445664 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.447882 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.447934 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.470456 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.472610 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.479662 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.480767 5411 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.480767 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.481209 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.481559 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.482995 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
I20260809 19:21:14.490494 4039 test_util.cc:268] Using random seed: 606280308
W20260809 19:21:14.504837 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.506464 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.516667 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.517575 5849 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.517712 5850 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.517634 5851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.517620 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.517719 5844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.541025 5844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.541944 5844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.554019 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.554037 6304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.554037 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.554864 6313 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.554916 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.554880 6304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.578307 6304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.581491 6304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.591408 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.591425 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.620041 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.629323 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.631279 8494 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.631301 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.633508 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.655967 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.667650 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.669818 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.669869 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.670866 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.672060 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.675094 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.694407 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.699262 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.709095 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.713409 7950 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.713420 7941 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.713409 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.714422 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.716604 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.738092 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.742319 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:14.752283 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.753993 5413 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.754109 5411 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.754109 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.756927 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.759680 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.779559 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.784468 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.794531 5844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.795415 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.795497 5844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.797556 5844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.798672 5844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.804981 5844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.823844 5844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.829037 5844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.838022 6304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.840024 6304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.841173 6304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.843405 6304 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.843489 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.846624 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.870078 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.872126 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:14.887831 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.888916 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
I20260809 19:21:14.893860 4039 test_util.cc:268] Using random seed: 606683678
W20260809 19:21:14.915827 6681 meta_cache.cc:302] tablet 7c092d80045840deac3151a94e01de0b: replica dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757) has failed: Network error: Client connection negotiation failed: client connection to 127.3.241.193:46757: connect: Connection refused (error 111)
W20260809 19:21:14.917436 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.927722 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.938809 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.964107 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.975849 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.981057 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.983202 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.983212 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.984303 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:14.984897 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.009546 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.013793 6529 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.026886 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:15.029904 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:15.030875 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:15.033178 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:15.034389 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:15.035482 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:15.056978 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:15.064137 7947 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47088: Illegal state: replica 61427f2036ac4f96a9655aef910a7d7a is not leader of this config: current role FOLLOWER
W20260809 19:21:15.078037 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.078084 8495 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:41606: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.079628 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.081197 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.081657 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.086306 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.106062 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.113981 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.126103 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.128880 5844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.131084 5844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.132259 5844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.132308 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.137682 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
I20260809 19:21:15.148646 8830 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:21:15.148747 8830 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:15.149055 8830 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:15.149328 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 12 candidate_status { last_received { term: 11 index: 3059 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:15.149322 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 12 candidate_status { last_received { term: 11 index: 3059 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:15.149426 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 11.
I20260809 19:21:15.149468 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 12 candidate_status { last_received { term: 11 index: 3059 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:15.149585 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 11.
I20260809 19:21:15.149573 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 12 candidate_status { last_received { term: 11 index: 3059 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:15.149789 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 12 candidate_status { last_received { term: 11 index: 3059 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
W20260809 19:21:15.149844 8440 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.193:46757: connect: Connection refused (error 111)
I20260809 19:21:15.149910 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 11.
I20260809 19:21:15.150099 8440 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a
I20260809 19:21:15.150187 8830 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260809 19:21:15.150257 8830 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 11 FOLLOWER]: Starting leader election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:21:15.150291 8830 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 11 FOLLOWER]: Advancing to term 12
W20260809 19:21:15.150424 8440 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757): Network error: Client connection negotiation failed: client connection to 127.3.241.193:46757: connect: Connection refused (error 111)
I20260809 19:21:15.151052 8830 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:15.151233 8830 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 12 election: Requested vote from peers 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:15.151376 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 12 candidate_status { last_received { term: 11 index: 3059 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:15.151383 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 12 candidate_status { last_received { term: 11 index: 3059 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:15.151446 5918 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 11 FOLLOWER]: Advancing to term 12
I20260809 19:21:15.151520 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 12 candidate_status { last_received { term: 11 index: 3059 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:15.151554 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 12 candidate_status { last_received { term: 11 index: 3059 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:15.151597 6381 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 11 FOLLOWER]: Advancing to term 12
I20260809 19:21:15.151651 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 12 candidate_status { last_received { term: 11 index: 3059 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:15.151721 5482 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 11 FOLLOWER]: Advancing to term 12
W20260809 19:21:15.152037 8440 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757): Network error: Client connection negotiation failed: client connection to 127.3.241.193:46757: connect: Connection refused (error 111)
I20260809 19:21:15.152326 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 12.
I20260809 19:21:15.152479 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 12.
I20260809 19:21:15.152647 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 12.
I20260809 19:21:15.152832 8440 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:15.152956 8830 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 12 FOLLOWER]: Leader election won for term 12
I20260809 19:21:15.153057 8830 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 12 LEADER]: Becoming Leader. State: Replica: 757ec8723e924451a89abb5cc62c1073, State: Running, Role: LEADER
I20260809 19:21:15.153133 8830 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3059, Committed index: 3059, Last appended: 11.3059, Last appended by leader: 3059, Current term: 12, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:15.154012 5043 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 reported cstate change: term changed from 11 to 12, leader changed from dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193) to 757ec8723e924451a89abb5cc62c1073 (127.3.241.197). New cstate: current_term: 12 leader_uuid: "757ec8723e924451a89abb5cc62c1073" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
W20260809 19:21:15.157755 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.163937 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.177690 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:15.179296 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:15.180789 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:15.181959 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:15.183075 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:15.189244 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:15.208789 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:15.213905 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
I20260809 19:21:15.230415 6381 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 12 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 9 index: 2767. Preceding OpId from leader: term: 12 index: 3061. (index mismatch)
I20260809 19:21:15.230418 5482 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 12 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 10 index: 3058. Preceding OpId from leader: term: 12 index: 3061. (index mismatch)
I20260809 19:21:15.230418 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 12 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 9 index: 2904. Preceding OpId from leader: term: 12 index: 3061. (index mismatch)
I20260809 19:21:15.230568 8005 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 11 FOLLOWER]: Advancing to term 12
I20260809 19:21:15.230589 6586 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 11 FOLLOWER]: Advancing to term 12
I20260809 19:21:15.230733 8845 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3060, Last known committed idx: 2899, Time since last communication: 0.000s
I20260809 19:21:15.231122 8842 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3060, Last known committed idx: 3058, Time since last communication: 0.000s
I20260809 19:21:15.231271 8842 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3060, Last known committed idx: 2766, Time since last communication: 0.000s
I20260809 19:21:15.231628 8005 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 12 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 11 index: 3059. Preceding OpId from leader: term: 12 index: 3061. (index mismatch)
I20260809 19:21:15.231632 6586 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 12 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 11 index: 3059. Preceding OpId from leader: term: 12 index: 3061. (index mismatch)
I20260809 19:21:15.231895 8845 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3060, Last known committed idx: 3059, Time since last communication: 0.000s
W20260809 19:21:15.232388 8440 consensus_peers.cc:487] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 -> Peer dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757): Couldn't send request to peer dd68273a4b314386ae0f5489e2f1d468. Status: Network error: Client connection negotiation failed: client connection to 127.3.241.193:46757: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 19:21:15.232497 8845 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3060, Last known committed idx: 3059, Time since last communication: 0.000s
I20260809 19:21:15.397621 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.193:46757
--local_ip_for_outbound_sockets=127.3.241.193
--tserver_master_addrs=127.3.241.254:44125
--webserver_port=33375
--webserver_interface=127.3.241.193
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260809 19:21:15.579592 8912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:15.580009 8912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:15.580113 8912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:15.582517 8912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:15.582782 8912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.193
I20260809 19:21:15.585233 8912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.193:46757
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.3.241.193
--webserver_port=33375
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.8912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.193
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:15.586208 8912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:15.586709 8912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:15.595084 8919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:15.595561 8922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:15.603286 8912 server_base.cc:1034] running on GCE node
W20260809 19:21:15.604516 8920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:15.605733 8912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:15.606078 8912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:15.613440 8912 hybrid_clock.cc:648] HybridClock initialized: now 1786303275612996 us; error 445 us; skew 500 ppm
I20260809 19:21:15.618007 8912 webserver.cc:460] Webserver started at http://127.3.241.193:33375/ using document root <none> and password file <none>
I20260809 19:21:15.618326 8912 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:15.618402 8912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:15.621671 8912 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:15.626122 8933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:15.627882 8912 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260809 19:21:15.628043 8912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "dd68273a4b314386ae0f5489e2f1d468"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:21:15.628679 8912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:15.666023 8912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:15.666648 8912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:15.666935 8912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:15.667375 8912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:15.668419 8946 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:21:15.671365 8912 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:21:15.671417 8912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260809 19:21:15.671454 8912 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:21:15.672587 8912 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:21:15.672706 8912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:15.680325 8946 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Bootstrap starting.
I20260809 19:21:15.687074 8912 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.193:46757
I20260809 19:21:15.688009 8912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260809 19:21:15.692267 9109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.193:46757 every 8 connection(s)
I20260809 19:21:15.698179 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 8912
I20260809 19:21:15.732278 9110 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:21:15.732398 9110 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:15.732599 9110 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:21:15.733631 5040 ts_manager.cc:194] Re-registered known tserver with Master: dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757)
I20260809 19:21:15.734167 5040 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.193:39745
I20260809 19:21:15.776669 8946 log.cc:824] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Log is configured to *not* fsync() on all Append() calls
W20260809 19:21:15.818390 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.820613 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.822918 5844 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.823077 5851 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.823467 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:15.823967 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
I20260809 19:21:16.088464 8946 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Bootstrap replayed 1/1 log segments. Stats: ops{read=3074 overwritten=15 applied=3059 ignored=0} inserts{seen=30490 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:21:16.088914 8946 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Bootstrap complete.
I20260809 19:21:16.090066 8946 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Time spent bootstrapping tablet: real 0.410s user 0.245s sys 0.112s
I20260809 19:21:16.091457 8946 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 11 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:16.091717 8946 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 11 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd68273a4b314386ae0f5489e2f1d468, State: Initialized, Role: FOLLOWER
I20260809 19:21:16.091871 8946 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3059, Last appended: 11.3059, Last appended by leader: 3059, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:16.092221 8946 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20260809 19:21:16.092281 9110 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
I20260809 19:21:16.110168 9014 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 11 FOLLOWER]: Advancing to term 12
I20260809 19:21:16.141283 9121 mvcc.cc:205] Tried to move back new op lower bound from 7316698218581536768 to 7316698215028137984. Current Snapshot: MvccSnapshot[applied={T|T < 7316698217922424832}]
I20260809 19:21:16.462862 4039 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKf1XAf/build/release/bin/kudu with pid 8337
W20260809 19:21:16.478997 6684 meta_cache.cc:302] tablet 7c092d80045840deac3151a94e01de0b: replica 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111) has failed: Network error: recv got EOF from 127.3.241.197:45111 (error 108)
W20260809 19:21:16.481170 6267 connection.cc:441] server connection from 127.3.241.197:53213 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.197:53213 (request call id 280) could send its response
W20260809 19:21:16.487600 5379 connection.cc:441] server connection from 127.3.241.197:35763 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.197:35763 (request call id 459) could send its response
W20260809 19:21:16.487923 5804 connection.cc:441] server connection from 127.3.241.197:34101 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.197:34101 (request call id 388) could send its response
I20260809 19:21:16.489856 4039 test_util.cc:268] Using random seed: 608279671
W20260809 19:21:16.491047 7905 connection.cc:441] server connection from 127.3.241.197:35445 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.197:35445 (request call id 475) could send its response
I20260809 19:21:16.593997 4039 test_util.cc:268] Using random seed: 608383811
I20260809 19:21:16.798285 4039 test_util.cc:268] Using random seed: 608588091
I20260809 19:21:17.102687 4039 test_util.cc:268] Using random seed: 608892501
I20260809 19:21:17.506705 4039 test_util.cc:268] Using random seed: 609296518
I20260809 19:21:17.635118 9130 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:17.635286 9130 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:17.635716 9130 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
W20260809 19:21:17.636108 8938 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
W20260809 19:21:17.638034 8938 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:17.639442 9157 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:17.639576 9157 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
W20260809 19:21:17.640275 6487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260809 19:21:17.640468 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 13 candidate_status { last_received { term: 12 index: 3918 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:17.640452 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 13 candidate_status { last_received { term: 12 index: 3918 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:17.640565 6586 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 12.
I20260809 19:21:17.640913 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 13 candidate_status { last_received { term: 12 index: 3918 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:17.641667 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 13 candidate_status { last_received { term: 12 index: 3918 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
W20260809 19:21:17.641722 6487 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:17.641933 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 13 candidate_status { last_received { term: 12 index: 3918 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:17.642005 6486 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 23183da38752444a9ff5525be50cd404; no voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506
I20260809 19:21:17.642195 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 13 candidate_status { last_received { term: 12 index: 3918 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:17.642355 9157 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:17.642426 9157 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260809 19:21:17.642673 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 13 candidate_status { last_received { term: 12 index: 3918 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:17.643237 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 13 candidate_status { last_received { term: 12 index: 3918 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:17.643404 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 13 candidate_status { last_received { term: 12 index: 3918 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:17.643611 8935 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 23183da38752444a9ff5525be50cd404, dd68273a4b314386ae0f5489e2f1d468; no voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad
I20260809 19:21:17.643749 9130 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260809 19:21:17.644747 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 13 candidate_status { last_received { term: 12 index: 3918 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:17.644892 9014 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 12.
I20260809 19:21:17.981601 9165 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:17.981742 9165 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:17.982041 9165 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545)
W20260809 19:21:17.982410 6271 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:17.982383 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 13 candidate_status { last_received { term: 12 index: 3685 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:17.982506 6586 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4c4b93755eb4aa48c04795ad08e09ad for term 13 because replica has last-logged OpId of term: 12 index: 3918, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 3685.
I20260809 19:21:17.982442 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 13 candidate_status { last_received { term: 12 index: 3685 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:17.982595 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 13 candidate_status { last_received { term: 12 index: 3685 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:17.982661 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 13 candidate_status { last_received { term: 12 index: 3685 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:17.983058 6267 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506
I20260809 19:21:17.983073 9172 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:17.983136 9172 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:17.983258 9165 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260809 19:21:17.983505 6271 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:17.983515 9172 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
W20260809 19:21:17.983839 5805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:17.983949 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 13 candidate_status { last_received { term: 12 index: 3798 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:17.984033 6586 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for term 13 because replica has last-logged OpId of term: 12 index: 3918, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 3798.
I20260809 19:21:17.984015 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 13 candidate_status { last_received { term: 12 index: 3798 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:17.984115 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 13 candidate_status { last_received { term: 12 index: 3798 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:17.984335 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 13 candidate_status { last_received { term: 12 index: 3798 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:17.984442 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 12.
W20260809 19:21:17.985142 5805 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:17.985198 5805 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506
I20260809 19:21:17.985281 9172 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260809 19:21:17.985407 9179 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:17.985499 9179 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:17.985942 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 13 candidate_status { last_received { term: 12 index: 3915 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:17.986016 9179 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:17.986032 6586 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 for term 13 because replica has last-logged OpId of term: 12 index: 3918, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 3915.
I20260809 19:21:17.986251 8005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 13 candidate_status { last_received { term: 12 index: 3915 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:17.986328 8005 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 for term 13 because replica has last-logged OpId of term: 12 index: 3917, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 3915.
I20260809 19:21:17.986418 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 13 candidate_status { last_received { term: 12 index: 3915 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:17.986487 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 12.
I20260809 19:21:17.986765 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 13 candidate_status { last_received { term: 12 index: 3915 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:17.986840 9188 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:17.986850 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 12.
I20260809 19:21:17.986900 9188 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:17.987188 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 13 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:17.987290 6586 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a for term 13 because replica has last-logged OpId of term: 12 index: 3918, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 3917.
I20260809 19:21:17.987566 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 13 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:17.987639 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 12.
I20260809 19:21:17.987661 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 13 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:17.987732 5479 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 12.
I20260809 19:21:17.987507 9188 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:17.987818 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 13 candidate_status { last_received { term: 12 index: 3798 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:17.987895 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 13 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:17.987959 9014 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 for term 13 because replica has last-logged OpId of term: 12 index: 3918, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 3798.
I20260809 19:21:17.987963 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 12.
W20260809 19:21:17.988201 7904 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260809 19:21:17.988317 7902 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404
I20260809 19:21:17.988503 9188 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260809 19:21:17.988619 9188 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:17.988682 9188 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 12 FOLLOWER]: Advancing to term 13
W20260809 19:21:17.989007 5379 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:17.989615 9188 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:17.989671 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 13 candidate_status { last_received { term: 12 index: 3685 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:17.989787 9014 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4c4b93755eb4aa48c04795ad08e09ad for term 13 because replica has last-logged OpId of term: 12 index: 3918, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 3685.
I20260809 19:21:17.989931 6586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 13 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:17.990005 6586 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 12 FOLLOWER]: Advancing to term 13
I20260809 19:21:17.989997 9188 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 13 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:17.990098 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 13 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:17.990166 5479 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 12 FOLLOWER]: Advancing to term 13
I20260809 19:21:17.990152 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 13 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:17.990206 6381 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 12 FOLLOWER]: Advancing to term 13
W20260809 19:21:17.990543 7904 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:17.990849 6586 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a for term 13 because replica has last-logged OpId of term: 12 index: 3918, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 3917.
I20260809 19:21:17.990916 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 13.
I20260809 19:21:17.990945 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 13 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:17.991004 5918 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 12 FOLLOWER]: Advancing to term 13
I20260809 19:21:17.990998 5479 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 13.
W20260809 19:21:17.991231 7904 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:17.991812 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 13.
I20260809 19:21:17.991956 7903 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 757ec8723e924451a89abb5cc62c1073
I20260809 19:21:17.992048 9188 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 13 FOLLOWER]: Leader election won for term 13
I20260809 19:21:17.992208 9188 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 13 LEADER]: Becoming Leader. State: Replica: 61427f2036ac4f96a9655aef910a7d7a, State: Running, Role: LEADER
I20260809 19:21:17.992339 9188 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3916, Committed index: 3916, Last appended: 12.3917, Last appended by leader: 3917, Current term: 13, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:17.993219 5042 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a reported cstate change: term changed from 12 to 13, leader changed from 757ec8723e924451a89abb5cc62c1073 (127.3.241.197) to 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195). New cstate: current_term: 13 leader_uuid: "61427f2036ac4f96a9655aef910a7d7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
I20260809 19:21:17.993505 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 13 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:17.993515 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 13 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:21:17.993606 9014 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a for term 13 because replica has last-logged OpId of term: 12 index: 3918, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 3917.
I20260809 19:21:17.994956 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 13 candidate_status { last_received { term: 12 index: 3915 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:17.995066 9014 raft_consensus.cc:2396] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d4177500c7a4f41afce5190ff786506 for term 13 because replica has last-logged OpId of term: 12 index: 3918, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 3915.
I20260809 19:21:17.995270 5375 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:17.995378 9179 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260809 19:21:18.010985 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.197:45111
--local_ip_for_outbound_sockets=127.3.241.197
--tserver_master_addrs=127.3.241.254:44125
--webserver_port=38231
--webserver_interface=127.3.241.197
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260809 19:21:18.097831 9212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:18.098027 9212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:18.098047 9212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:18.099210 9212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:18.099265 9212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.197
I20260809 19:21:18.100672 9212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.197:45111
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.3.241.197
--webserver_port=38231
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.9212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.197
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:18.100936 9212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:18.101248 9212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:18.106530 9222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:18.106530 9219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:18.106547 9220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:18.106971 9212 server_base.cc:1034] running on GCE node
I20260809 19:21:18.107252 9212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:18.107532 9212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:18.108704 9212 hybrid_clock.cc:648] HybridClock initialized: now 1786303278108684 us; error 31 us; skew 500 ppm
I20260809 19:21:18.110085 9212 webserver.cc:460] Webserver started at http://127.3.241.197:38231/ using document root <none> and password file <none>
I20260809 19:21:18.110347 9212 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:18.110427 9212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:18.111867 9212 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 19:21:18.112637 9234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.112859 9212 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:21:18.112982 9212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "757ec8723e924451a89abb5cc62c1073"
format_stamp: "Formatted at 2026-08-09 19:20:49 on dist-test-slave-gmjp"
I20260809 19:21:18.113350 9212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:18.132726 9212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:18.133087 9212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:18.133273 9212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:18.133553 9212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:18.134083 9244 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:21:18.135380 9212 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:21:18.135466 9212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:21:18.135519 9212 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:21:18.136301 9212 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:21:18.136355 9212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260809 19:21:18.136426 9244 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Bootstrap starting.
I20260809 19:21:18.144129 9212 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.197:45111
I20260809 19:21:18.144224 9421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.197:45111 every 8 connection(s)
I20260809 19:21:18.144594 9212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20260809 19:21:18.147894 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 9212
I20260809 19:21:18.153635 9422 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:21:18.153759 9422 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:18.153972 9422 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:21:18.154687 5042 ts_manager.cc:194] Re-registered known tserver with Master: 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111)
I20260809 19:21:18.155181 5042 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.197:35047
I20260809 19:21:18.200888 9244 log.cc:824] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:18.452198 9244 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Bootstrap replayed 1/1 log segments. Stats: ops{read=3940 overwritten=22 applied=3916 ignored=0} inserts{seen=39050 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20260809 19:21:18.452684 9244 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Bootstrap complete.
I20260809 19:21:18.453923 9244 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Time spent bootstrapping tablet: real 0.318s user 0.207s sys 0.108s
I20260809 19:21:18.454879 9244 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 12 FOLLOWER]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:18.455637 9244 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 12 FOLLOWER]: Becoming Follower/Learner. State: Replica: 757ec8723e924451a89abb5cc62c1073, State: Initialized, Role: FOLLOWER
I20260809 19:21:18.455842 9244 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3916, Last appended: 12.3918, Last appended by leader: 3918, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:18.456197 9244 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20260809 19:21:18.456223 9422 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
I20260809 19:21:18.533556 9353 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 12 FOLLOWER]: Advancing to term 13
I20260809 19:21:18.534775 9353 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 13 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 12 index: 3918. Preceding OpId from leader: term: 13 index: 3918. (term mismatch)
I20260809 19:21:18.534821 9353 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting all ops after (but not including) 3917
I20260809 19:21:18.534844 9353 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted WRITE_OP operation due to leader change: 12.3918
I20260809 19:21:18.535259 9188 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3918, Last known committed idx: 3916, Time since last communication: 0.000s
I20260809 19:21:18.827979 4039 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKf1XAf/build/release/bin/kudu with pid 7877
I20260809 19:21:18.839640 6381 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 13 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 12 index: 3685. Preceding OpId from leader: term: 13 index: 3918. (index mismatch)
W20260809 19:21:18.839829 6267 connection.cc:441] server connection from 127.3.241.195:37207 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.195:37207 (request call id 5) could send its response
I20260809 19:21:18.841171 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 13 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 12 index: 3798. Preceding OpId from leader: term: 13 index: 3918. (index mismatch)
W20260809 19:21:18.841441 5805 connection.cc:441] server connection from 127.3.241.195:52931 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.195:52931 (request call id 5) could send its response
I20260809 19:21:18.842504 5479 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 13 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 12 index: 3915. Preceding OpId from leader: term: 13 index: 3918. (index mismatch)
W20260809 19:21:18.842592 5377 connection.cc:441] server connection from 127.3.241.195:59283 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.195:59283 (request call id 5) could send its response
I20260809 19:21:18.845934 9014 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 12 FOLLOWER]: Advancing to term 13
I20260809 19:21:18.846386 6586 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 13 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 12 index: 3918. Preceding OpId from leader: term: 13 index: 3918. (term mismatch)
I20260809 19:21:18.846441 6586 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting all ops after (but not including) 3917
I20260809 19:21:18.846462 6586 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Aborting uncommitted WRITE_OP operation due to leader change: 12.3918
I20260809 19:21:18.846575 6586 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=3905, attempt_no=1}) took 2358 ms. Trace:
I20260809 19:21:18.846619 6586 rpcz_store.cc:276] 0809 19:21:16.488183 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:16.488213 (+ 30us) service_pool.cc:227] Handling call
0809 19:21:18.846571 (+2358358us) inbound_call.cc:173] Queueing failure response
Metrics: {}
W20260809 19:21:18.846772 6486 connection.cc:441] server connection from 127.3.241.195:36209 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.195:36209 (request call id 6) could send its response
I20260809 19:21:18.846954 9014 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 13 FOLLOWER]: Refusing update from remote peer 61427f2036ac4f96a9655aef910a7d7a: Log matching property violated. Preceding OpId in replica: term: 12 index: 3918. Preceding OpId from leader: term: 13 index: 3918. (term mismatch)
I20260809 19:21:18.847004 9014 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting all ops after (but not including) 3917
I20260809 19:21:18.847047 4039 test_util.cc:268] Using random seed: 610636864
I20260809 19:21:18.847026 9014 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468: Aborting uncommitted WRITE_OP operation due to leader change: 12.3918
W20260809 19:21:18.847328 8934 connection.cc:441] server connection from 127.3.241.195:44685 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.195:44685 (request call id 2) could send its response
W20260809 19:21:18.853299 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:18.858631 8965 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:52680: Illegal state: replica dd68273a4b314386ae0f5489e2f1d468 is not leader of this config: current role FOLLOWER
W20260809 19:21:18.866839 6681 meta_cache.cc:302] tablet 7c092d80045840deac3151a94e01de0b: replica 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997) has failed: Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
W20260809 19:21:18.873456 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:18.882367 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:18.892760 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:18.918081 9275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40920: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:18.929076 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:18.943022 8965 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:52680: Illegal state: replica dd68273a4b314386ae0f5489e2f1d468 is not leader of this config: current role FOLLOWER
I20260809 19:21:18.950708 4039 test_util.cc:268] Using random seed: 610740523
W20260809 19:21:18.957837 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:18.971827 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:18.988003 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:19.025768 9275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40920: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:19.043787 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:19.062809 8965 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:52680: Illegal state: replica dd68273a4b314386ae0f5489e2f1d468 is not leader of this config: current role FOLLOWER
W20260809 19:21:19.082834 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:19.104876 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:19.131096 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
I20260809 19:21:19.154853 4039 test_util.cc:268] Using random seed: 610944668
W20260809 19:21:19.182121 9275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40920: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:19.208128 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
W20260809 19:21:19.236287 8965 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:52680: Illegal state: replica dd68273a4b314386ae0f5489e2f1d468 is not leader of this config: current role FOLLOWER
W20260809 19:21:19.267285 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:19.297361 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:19.326813 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
W20260809 19:21:19.388543 9275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40920: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
W20260809 19:21:19.424790 6525 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:43184: Illegal state: replica 23183da38752444a9ff5525be50cd404 is not leader of this config: current role FOLLOWER
I20260809 19:21:19.458844 4039 test_util.cc:268] Using random seed: 611248655
W20260809 19:21:19.458855 8965 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:52680: Illegal state: replica dd68273a4b314386ae0f5489e2f1d468 is not leader of this config: current role FOLLOWER
W20260809 19:21:19.496735 5410 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53568: Illegal state: replica 7d4177500c7a4f41afce5190ff786506 is not leader of this config: current role FOLLOWER
W20260809 19:21:19.533989 5846 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56638: Illegal state: replica 6c075d51984240f6b5d45d0d25a0d595 is not leader of this config: current role FOLLOWER
W20260809 19:21:19.574106 6311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:59094: Illegal state: replica d4c4b93755eb4aa48c04795ad08e09ad is not leader of this config: current role FOLLOWER
I20260809 19:21:19.623826 9441 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:19.624056 9441 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:19.624380 9441 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:19.624621 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 14 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:19.624662 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 14 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:19.624742 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 13.
I20260809 19:21:19.624769 9014 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 13.
I20260809 19:21:19.624794 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 14 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:19.624887 5479 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 13.
I20260809 19:21:19.625133 6484 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468; no voters:
I20260809 19:21:19.625101 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 14 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:19.625196 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 13.
I20260809 19:21:19.625219 9441 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260809 19:21:19.625257 9441 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:21:19.625284 9441 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 13 FOLLOWER]: Advancing to term 14
W20260809 19:21:19.625602 6484 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997): Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:21:19.626135 9441 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:19.626320 9441 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 14 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:19.626446 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 14 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:21:19.626477 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 14 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:19.626525 9014 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 13 FOLLOWER]: Advancing to term 14
I20260809 19:21:19.626555 5918 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 13 FOLLOWER]: Advancing to term 14
I20260809 19:21:19.626582 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 14 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:19.626650 5479 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 13 FOLLOWER]: Advancing to term 14
I20260809 19:21:19.627301 9014 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 14.
I20260809 19:21:19.627318 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 14.
I20260809 19:21:19.627405 5479 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 14.
I20260809 19:21:19.627547 6484 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468; no voters:
I20260809 19:21:19.627532 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 14 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:19.627606 6381 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 13 FOLLOWER]: Advancing to term 14
I20260809 19:21:19.627729 9441 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 14 FOLLOWER]: Leader election won for term 14
I20260809 19:21:19.627861 9441 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 14 LEADER]: Becoming Leader. State: Replica: 23183da38752444a9ff5525be50cd404, State: Running, Role: LEADER
I20260809 19:21:19.627969 9441 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3916, Committed index: 3916, Last appended: 12.3917, Last appended by leader: 3918, Current term: 14, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
W20260809 19:21:19.628130 6484 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997): Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111)
I20260809 19:21:19.628434 6381 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 14.
I20260809 19:21:19.628687 9352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 14 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073"
I20260809 19:21:19.628674 9353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 14 candidate_status { last_received { term: 12 index: 3917 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:19.629263 5037 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 reported cstate change: term changed from 13 to 14, leader changed from 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195) to 23183da38752444a9ff5525be50cd404 (127.3.241.199). New cstate: current_term: 14 leader_uuid: "23183da38752444a9ff5525be50cd404" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
W20260809 19:21:19.652660 9275 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40920: Illegal state: replica 757ec8723e924451a89abb5cc62c1073 is not leader of this config: current role FOLLOWER
I20260809 19:21:19.694389 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 14 FOLLOWER]: Refusing update from remote peer 23183da38752444a9ff5525be50cd404: Log matching property violated. Preceding OpId in replica: term: 12 index: 3798. Preceding OpId from leader: term: 14 index: 3919. (index mismatch)
I20260809 19:21:19.694435 9353 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 13 FOLLOWER]: Advancing to term 14
I20260809 19:21:19.694427 5479 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 14 FOLLOWER]: Refusing update from remote peer 23183da38752444a9ff5525be50cd404: Log matching property violated. Preceding OpId in replica: term: 12 index: 3915. Preceding OpId from leader: term: 14 index: 3919. (index mismatch)
I20260809 19:21:19.694664 6381 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 14 FOLLOWER]: Refusing update from remote peer 23183da38752444a9ff5525be50cd404: Log matching property violated. Preceding OpId in replica: term: 12 index: 3685. Preceding OpId from leader: term: 14 index: 3919. (index mismatch)
I20260809 19:21:19.694721 9014 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 14 FOLLOWER]: Refusing update from remote peer 23183da38752444a9ff5525be50cd404: Log matching property violated. Preceding OpId in replica: term: 12 index: 3917. Preceding OpId from leader: term: 14 index: 3919. (index mismatch)
I20260809 19:21:19.694933 9451 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3918, Last known committed idx: 3796, Time since last communication: 0.000s
I20260809 19:21:19.695202 9441 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3918, Last known committed idx: 3680, Time since last communication: 0.000s
W20260809 19:21:19.695333 6484 consensus_peers.cc:487] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 -> Peer 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997): Couldn't send request to peer 61427f2036ac4f96a9655aef910a7d7a. Status: Network error: Client connection negotiation failed: client connection to 127.3.241.195:36997: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 19:21:19.695479 9465 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3918, Last known committed idx: 3914, Time since last communication: 0.000s
I20260809 19:21:19.695557 9451 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3918, Last known committed idx: 3916, Time since last communication: 0.000s
I20260809 19:21:19.695597 9353 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 14 FOLLOWER]: Refusing update from remote peer 23183da38752444a9ff5525be50cd404: Log matching property violated. Preceding OpId in replica: term: 13 index: 3918. Preceding OpId from leader: term: 14 index: 3919. (index mismatch)
I20260809 19:21:19.695977 9464 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [LEADER]: Connected to new peer: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3918, Last known committed idx: 3916, Time since last communication: 0.000s
I20260809 19:21:19.696081 9464 consensus_queue.cc:1225] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [LEADER]: Peer 757ec8723e924451a89abb5cc62c1073 log is divergent from this leader: its last log entry 13.3918 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3916
I20260809 19:21:19.696599 9353 raft_consensus.cc:1203] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 12.3916->[12.3917-14.3919] Dedup: 12.3917->[14.3918-14.3919]
I20260809 19:21:19.696666 9353 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting all ops after (but not including) 3917
I20260809 19:21:19.696683 9353 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Aborting uncommitted NO_OP operation due to leader change: 13.3918
I20260809 19:21:19.696705 9353 raft_consensus.cc:2866] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 19:21:19.703305 9483 mvcc.cc:205] Tried to move back new op lower bound from 7316698233625944064 to 7316698233358901248. Current Snapshot: MvccSnapshot[applied={T|T < 7316698219031642112}]
I20260809 19:21:19.703744 9508 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:43184 (ReqId={client: 4cb4bff957a3440b94a2f9639d169335, seq_no=3906, attempt_no=1}) took 3215 ms. Trace:
I20260809 19:21:19.703897 9508 rpcz_store.cc:276] 0809 19:21:16.488136 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 19:21:16.488186 (+ 50us) service_pool.cc:227] Handling call
0809 19:21:19.703727 (+3215541us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 19:21:19.862874 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.195:36997
--local_ip_for_outbound_sockets=127.3.241.195
--tserver_master_addrs=127.3.241.254:44125
--webserver_port=42257
--webserver_interface=127.3.241.195
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260809 19:21:19.950037 9526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:19.950279 9526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:19.950316 9526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:19.951509 9526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:19.951587 9526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.195
I20260809 19:21:19.952999 9526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.195:36997
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.3.241.195
--webserver_port=42257
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.9526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.195
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:19.953256 9526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:19.953576 9526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:19.958776 9533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:19.958776 9538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:19.958802 9534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:19.959353 9526 server_base.cc:1034] running on GCE node
I20260809 19:21:19.959555 9526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:19.959748 9526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:19.960891 9526 hybrid_clock.cc:648] HybridClock initialized: now 1786303279960876 us; error 29 us; skew 500 ppm
I20260809 19:21:19.962352 9526 webserver.cc:460] Webserver started at http://127.3.241.195:42257/ using document root <none> and password file <none>
I20260809 19:21:19.962621 9526 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:19.962670 9526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:19.964068 9526 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:19.964792 9550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:19.964970 9526 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:19.965075 9526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "61427f2036ac4f96a9655aef910a7d7a"
format_stamp: "Formatted at 2026-08-09 19:20:48 on dist-test-slave-gmjp"
I20260809 19:21:19.965430 9526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:19.976195 9526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:19.976539 9526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:19.976696 9526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:19.976945 9526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:19.977424 9560 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:21:19.978701 9526 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:21:19.978770 9526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260809 19:21:19.978817 9526 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:21:19.979545 9526 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:21:19.979601 9526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 19:21:19.979666 9560 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Bootstrap starting.
I20260809 19:21:19.986652 9526 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.195:36997
I20260809 19:21:19.986742 9720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.195:36997 every 8 connection(s)
I20260809 19:21:19.987074 9526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260809 19:21:19.989465 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 9526
I20260809 19:21:19.999276 9721 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:21:19.999377 9721 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:19.999577 9721 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:21:20.000239 5037 ts_manager.cc:194] Re-registered known tserver with Master: 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997)
I20260809 19:21:20.000842 5037 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.195:37805
I20260809 19:21:20.047684 9560 log.cc:824] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:20.295881 9560 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Bootstrap replayed 1/1 log segments. Stats: ops{read=3928 overwritten=10 applied=3916 ignored=0} inserts{seen=39050 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20260809 19:21:20.296458 9560 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Bootstrap complete.
I20260809 19:21:20.297657 9560 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Time spent bootstrapping tablet: real 0.318s user 0.203s sys 0.112s
I20260809 19:21:20.298915 9560 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 13 FOLLOWER]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:20.299639 9560 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 13 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61427f2036ac4f96a9655aef910a7d7a, State: Initialized, Role: FOLLOWER
I20260809 19:21:20.299836 9560 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3916, Last appended: 13.3918, Last appended by leader: 3918, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:20.300125 9560 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260809 19:21:20.300204 9721 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
I20260809 19:21:20.691179 4039 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKf1XAf/build/release/bin/kudu with pid 6462
W20260809 19:21:20.705541 6271 connection.cc:441] server connection from 127.3.241.199:44217 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.199:44217 (request call id 152) could send its response
W20260809 19:21:20.706509 5803 connection.cc:441] server connection from 127.3.241.199:43021 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.199:43021 (request call id 200) could send its response
W20260809 19:21:20.708454 5375 connection.cc:441] server connection from 127.3.241.199:57587 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.199:57587 (request call id 267) could send its response
I20260809 19:21:20.711684 4039 test_util.cc:268] Using random seed: 612501501
I20260809 19:21:20.816031 4039 test_util.cc:268] Using random seed: 612605845
I20260809 19:21:21.020406 4039 test_util.cc:268] Using random seed: 612810220
I20260809 19:21:21.324649 4039 test_util.cc:268] Using random seed: 613114463
I20260809 19:21:21.728838 4039 test_util.cc:268] Using random seed: 613518653
I20260809 19:21:21.760090 9735 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 23183da38752444a9ff5525be50cd404)
I20260809 19:21:21.760349 9735 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:21.760772 9735 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
W20260809 19:21:21.761324 9236 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.199:41771: connect: Connection refused (error 111)
W20260809 19:21:21.764125 9236 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771): Network error: Client connection negotiation failed: client connection to 127.3.241.199:41771: connect: Connection refused (error 111)
I20260809 19:21:21.765594 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:21.765676 9654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:21.765858 9654 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 13.
I20260809 19:21:21.765851 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:21.766808 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:21.767063 9236 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073; no voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:21.767208 9735 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260809 19:21:21.768107 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:21.970098 9756 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 23183da38752444a9ff5525be50cd404)
I20260809 19:21:21.970214 9756 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:21.970538 9756 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:21.971099 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:21.971184 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:21.971303 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
W20260809 19:21:21.971774 8935 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771): Network error: Client connection negotiation failed: client connection to 127.3.241.199:41771: connect: Connection refused (error 111)
I20260809 19:21:21.971833 8935 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: dd68273a4b314386ae0f5489e2f1d468; no voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad
I20260809 19:21:21.971944 9756 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260809 19:21:21.974171 9654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:21.974280 9654 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 13.
I20260809 19:21:21.974547 9352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:21.974642 9352 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 14.
I20260809 19:21:22.206079 9765 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 23183da38752444a9ff5525be50cd404)
I20260809 19:21:22.206312 9765 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:22.206646 9765 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545)
I20260809 19:21:22.207037 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:22.207154 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 14.
I20260809 19:21:22.207094 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:22.207201 9014 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 14.
I20260809 19:21:22.207523 9769 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 23183da38752444a9ff5525be50cd404)
I20260809 19:21:22.207532 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:22.207595 9769 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:22.207649 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 14.
I20260809 19:21:22.207808 6267 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters:
I20260809 19:21:22.207878 9769 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
W20260809 19:21:22.208039 6270 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771): Network error: Client connection negotiation failed: client connection to 127.3.241.199:41771: connect: Connection refused (error 111)
I20260809 19:21:22.208141 9765 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260809 19:21:22.208200 9765 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 14 FOLLOWER]: Starting leader election (detected failure of leader 23183da38752444a9ff5525be50cd404)
I20260809 19:21:22.208223 9765 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 14 FOLLOWER]: Advancing to term 15
I20260809 19:21:22.208571 6381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:22.208565 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:22.208664 9014 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 14.
I20260809 19:21:22.208936 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:22.209026 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 14.
I20260809 19:21:22.209529 9765 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:22.209944 9765 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 15 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545)
I20260809 19:21:22.210083 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:22.210156 5482 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 14 FOLLOWER]: Advancing to term 15
I20260809 19:21:22.210294 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:22.210374 5918 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 14 FOLLOWER]: Advancing to term 15
I20260809 19:21:22.210374 6381 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 15: Already voted for candidate d4c4b93755eb4aa48c04795ad08e09ad in this term.
W20260809 19:21:22.210811 6270 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771): Network error: Client connection negotiation failed: client connection to 127.3.241.199:41771: connect: Connection refused (error 111)
I20260809 19:21:22.210976 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 15.
I20260809 19:21:22.211123 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:21:22.211199 9014 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 14 FOLLOWER]: Advancing to term 15
I20260809 19:21:22.211222 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 15.
I20260809 19:21:22.211663 9352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:22.211746 9352 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 14.
I20260809 19:21:22.211827 9353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073"
I20260809 19:21:22.211862 9353 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 14 FOLLOWER]: Advancing to term 15
I20260809 19:21:22.211985 9014 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 15.
I20260809 19:21:22.212208 6267 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters: 23183da38752444a9ff5525be50cd404
I20260809 19:21:22.212289 9765 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 15 FOLLOWER]: Leader election won for term 15
I20260809 19:21:22.212421 9765 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 15 LEADER]: Becoming Leader. State: Replica: d4c4b93755eb4aa48c04795ad08e09ad, State: Running, Role: LEADER
W20260809 19:21:22.212446 5804 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771): Network error: Client connection negotiation failed: client connection to 127.3.241.199:41771: connect: Connection refused (error 111)
I20260809 19:21:22.212532 9765 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4012, Committed index: 4012, Last appended: 14.4012, Last appended by leader: 4012, Current term: 15, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:22.212706 9353 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 15.
I20260809 19:21:22.213961 5037 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad reported cstate change: term changed from 14 to 15, leader changed from 23183da38752444a9ff5525be50cd404 (127.3.241.199) to d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198). New cstate: current_term: 15 leader_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: HEALTHY } } }
I20260809 19:21:22.214224 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:22.214331 9652 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 13 FOLLOWER]: Advancing to term 15
I20260809 19:21:22.214203 9654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:22.215328 9353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:22.215435 9353 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 15: Already voted for candidate d4c4b93755eb4aa48c04795ad08e09ad in this term.
I20260809 19:21:22.215883 9652 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 15.
I20260809 19:21:22.216751 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 15 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:22.216845 9652 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 15: Already voted for candidate d4c4b93755eb4aa48c04795ad08e09ad in this term.
I20260809 19:21:22.216997 5803 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad
I20260809 19:21:22.217098 9769 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260809 19:21:22.233225 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.199:41771
--local_ip_for_outbound_sockets=127.3.241.199
--tserver_master_addrs=127.3.241.254:44125
--webserver_port=44661
--webserver_interface=127.3.241.199
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260809 19:21:22.321889 9791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:22.322110 9791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:22.322131 9791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:22.323328 9791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:22.323387 9791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.199
I20260809 19:21:22.324870 9791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.199:41771
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.3.241.199
--webserver_port=44661
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.9791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.199
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:22.325150 9791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:22.325489 9791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:22.330646 9805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:22.330720 9802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:22.330646 9800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:22.331017 9791 server_base.cc:1034] running on GCE node
I20260809 19:21:22.331272 9791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:22.331552 9791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:22.332731 9791 hybrid_clock.cc:648] HybridClock initialized: now 1786303282332719 us; error 37 us; skew 500 ppm
I20260809 19:21:22.334019 9791 webserver.cc:460] Webserver started at http://127.3.241.199:44661/ using document root <none> and password file <none>
I20260809 19:21:22.334286 9791 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:22.334372 9791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:22.335744 9791 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:22.336465 9814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:22.336656 9791 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:22.336747 9791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "23183da38752444a9ff5525be50cd404"
format_stamp: "Formatted at 2026-08-09 19:20:49 on dist-test-slave-gmjp"
I20260809 19:21:22.337082 9791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:22.370702 9791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:22.371080 9791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:22.371248 9791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:22.371529 9791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:22.372084 9821 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:21:22.373438 9791 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:21:22.373507 9791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:21:22.373555 9791 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:21:22.374209 9791 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:21:22.374267 9791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 19:21:22.374333 9821 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Bootstrap starting.
I20260809 19:21:22.382210 9791 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.199:41771
I20260809 19:21:22.382301 9986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.199:41771 every 8 connection(s)
I20260809 19:21:22.382628 9791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20260809 19:21:22.389346 9987 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:21:22.389462 9987 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:22.389674 9987 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:21:22.390362 5037 ts_manager.cc:194] Re-registered known tserver with Master: 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771)
I20260809 19:21:22.390918 5037 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.199:59507
I20260809 19:21:22.391222 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 9791
I20260809 19:21:22.425520 9821 log.cc:824] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:22.642738 9014 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 15 FOLLOWER]: Refusing update from remote peer d4c4b93755eb4aa48c04795ad08e09ad: Log matching property violated. Preceding OpId in replica: term: 14 index: 4012. Preceding OpId from leader: term: 15 index: 4013. (index mismatch)
I20260809 19:21:22.643128 9778 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4013, Last known committed idx: 4012, Time since last communication: 0.000s
I20260809 19:21:22.675561 9821 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Bootstrap replayed 1/1 log segments. Stats: ops{read=4035 overwritten=23 applied=4012 ignored=0} inserts{seen=40000 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:21:22.676033 9821 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Bootstrap complete.
I20260809 19:21:22.677337 9821 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Time spent bootstrapping tablet: real 0.303s user 0.217s sys 0.082s
I20260809 19:21:22.678682 9353 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 15 FOLLOWER]: Refusing update from remote peer d4c4b93755eb4aa48c04795ad08e09ad: Log matching property violated. Preceding OpId in replica: term: 14 index: 4012. Preceding OpId from leader: term: 15 index: 4013. (index mismatch)
I20260809 19:21:22.679025 9778 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4013, Last known committed idx: 4012, Time since last communication: 0.000s
I20260809 19:21:22.679003 9821 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 14 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:22.679365 9821 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 14 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23183da38752444a9ff5525be50cd404, State: Initialized, Role: FOLLOWER
I20260809 19:21:22.679561 9821 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4012, Last appended: 14.4012, Last appended by leader: 4012, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:22.679941 9821 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260809 19:21:22.680083 9987 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
I20260809 19:21:22.757836 9916 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 14 FOLLOWER]: Advancing to term 15
I20260809 19:21:22.759330 9916 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 15 FOLLOWER]: Refusing update from remote peer d4c4b93755eb4aa48c04795ad08e09ad: Log matching property violated. Preceding OpId in replica: term: 14 index: 4012. Preceding OpId from leader: term: 15 index: 4013. (index mismatch)
I20260809 19:21:22.759766 9778 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4013, Last known committed idx: 4012, Time since last communication: 0.000s
I20260809 19:21:23.068998 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 15 FOLLOWER]: Refusing update from remote peer d4c4b93755eb4aa48c04795ad08e09ad: Log matching property violated. Preceding OpId in replica: term: 14 index: 4012. Preceding OpId from leader: term: 15 index: 4013. (index mismatch)
I20260809 19:21:23.073010 5482 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 15 FOLLOWER]: Refusing update from remote peer d4c4b93755eb4aa48c04795ad08e09ad: Log matching property violated. Preceding OpId in replica: term: 14 index: 4012. Preceding OpId from leader: term: 15 index: 4013. (index mismatch)
I20260809 19:21:23.073383 9652 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 15 FOLLOWER]: Refusing update from remote peer d4c4b93755eb4aa48c04795ad08e09ad: Log matching property violated. Preceding OpId in replica: term: 13 index: 3918. Preceding OpId from leader: term: 15 index: 4013. (index mismatch)
I20260809 19:21:23.175246 4039 test_util.cc:268] Using random seed: 614965057
I20260809 19:21:23.480535 4039 test_util.cc:268] Using random seed: 615270347
I20260809 19:21:23.885960 4039 test_util.cc:268] Using random seed: 615675772
I20260809 19:21:23.897730 10021 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:23.897828 10021 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:23.898178 10021 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:23.898556 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 16 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:23.898718 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 16 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:23.898821 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 15.
I20260809 19:21:23.901926 9353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 16 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:23.902629 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 16 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:23.902740 9652 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 15.
I20260809 19:21:23.902706 9916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 16 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:24.262074 10029 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d4c4b93755eb4aa48c04795ad08e09ad)
I20260809 19:21:24.262240 10033 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d4c4b93755eb4aa48c04795ad08e09ad)
I20260809 19:21:24.262248 10029 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:24.262362 10033 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:24.262661 10033 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:24.262996 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:24.263058 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:24.263145 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 15.
I20260809 19:21:24.263098 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:24.263203 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 15.
I20260809 19:21:24.263197 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:24.263290 9652 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 15.
I20260809 19:21:24.263422 9235 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506; no voters:
I20260809 19:21:24.263556 10033 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260809 19:21:24.263664 10033 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d4c4b93755eb4aa48c04795ad08e09ad)
I20260809 19:21:24.263690 10033 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 15 FOLLOWER]: Advancing to term 16
I20260809 19:21:24.264842 10029 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:24.265246 10033 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:24.266929 10033 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 16 election: Requested vote from peers 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:24.267144 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:24.267181 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:24.267236 5918 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 15 FOLLOWER]: Advancing to term 16
I20260809 19:21:24.267272 5482 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 15 FOLLOWER]: Advancing to term 16
I20260809 19:21:24.267233 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:24.267313 9652 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 15 FOLLOWER]: Advancing to term 16
I20260809 19:21:24.267562 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:21:24.268078 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 16.
I20260809 19:21:24.268077 9652 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 16.
I20260809 19:21:24.268133 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 16.
I20260809 19:21:24.268110 9916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:24.268220 9916 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 15.
I20260809 19:21:24.268265 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:24.268340 9915 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 15 FOLLOWER]: Advancing to term 16
I20260809 19:21:24.268481 9235 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506; no voters: dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:24.268559 10033 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 16 FOLLOWER]: Leader election won for term 16
I20260809 19:21:24.268671 10033 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 16 LEADER]: Becoming Leader. State: Replica: 757ec8723e924451a89abb5cc62c1073, State: Running, Role: LEADER
I20260809 19:21:24.268779 10033 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4013, Committed index: 4013, Last appended: 15.4013, Last appended by leader: 4013, Current term: 16, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:24.269198 9915 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 16.
I20260809 19:21:24.269721 9353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:24.269790 5037 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 reported cstate change: term changed from 15 to 16, leader changed from d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198) to 757ec8723e924451a89abb5cc62c1073 (127.3.241.197). New cstate: current_term: 16 leader_uuid: "757ec8723e924451a89abb5cc62c1073" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
I20260809 19:21:24.270063 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:24.270149 9652 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23183da38752444a9ff5525be50cd404 in current term 16: Already voted for candidate 757ec8723e924451a89abb5cc62c1073 in this term.
I20260809 19:21:24.271194 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:24.273571 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:24.273669 5918 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23183da38752444a9ff5525be50cd404 in current term 16: Already voted for candidate 757ec8723e924451a89abb5cc62c1073 in this term.
I20260809 19:21:24.273885 9816 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 23183da38752444a9ff5525be50cd404; no voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:24.274060 10029 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260809 19:21:24.274602 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:24.274718 5482 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23183da38752444a9ff5525be50cd404 in current term 16: Already voted for candidate 757ec8723e924451a89abb5cc62c1073 in this term.
I20260809 19:21:24.292407 6382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 16 candidate_status { last_received { term: 14 index: 4012 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:24.292701 5375 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506; no voters: 23183da38752444a9ff5525be50cd404, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:24.292632 9765 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4013, Last known committed idx: 3916, Time since last communication: 0.000s
I20260809 19:21:24.292714 6371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:24.292762 9765 consensus_queue.cc:1225] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Peer 61427f2036ac4f96a9655aef910a7d7a log is divergent from this leader: its last log entry 13.3918 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3916
I20260809 19:21:24.292806 10021 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260809 19:21:24.292851 6372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:24.292974 9778 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4013, Last known committed idx: 4012, Time since last communication: 0.000s
I20260809 19:21:24.293613 5918 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 16 FOLLOWER]: Rejecting Update request from peer d4c4b93755eb4aa48c04795ad08e09ad for earlier term 15. Current term is 16. Ops: [15.4013-15.4013]
I20260809 19:21:24.293720 9353 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 16 LEADER]: Rejecting Update request from peer d4c4b93755eb4aa48c04795ad08e09ad for earlier term 15. Current term is 16. Ops: []
I20260809 19:21:24.293893 9915 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 16 FOLLOWER]: Rejecting Update request from peer d4c4b93755eb4aa48c04795ad08e09ad for earlier term 15. Current term is 16. Ops: []
I20260809 19:21:24.294021 9765 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: INVALID_TERM, Last received: 15.4013, Next index: 4014, Last known committed idx: 4013, Time since last communication: 0.000s
I20260809 19:21:24.294123 9765 consensus_queue.cc:1225] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Peer 757ec8723e924451a89abb5cc62c1073 log is divergent from this leader: its last log entry 16.4014 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4013
I20260809 19:21:24.294179 10060 raft_consensus.cc:3032] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 15 LEADER]: Stepping down as leader of term 15
I20260809 19:21:24.294260 10060 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d4c4b93755eb4aa48c04795ad08e09ad, State: Running, Role: LEADER
I20260809 19:21:24.294346 10060 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4013, Committed index: 4013, Last appended: 15.4013, Last appended by leader: 4013, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:24.294489 10013 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: INVALID_TERM, Last received: 15.4013, Next index: 4014, Last known committed idx: 4013, Time since last communication: 0.000s
I20260809 19:21:24.294605 10013 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: INVALID_TERM, Last received: 14.4012, Next index: 4013, Last known committed idx: 4012, Time since last communication: 0.000s
I20260809 19:21:24.294761 10061 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4013, Last known committed idx: 4012, Time since last communication: 0.000s
I20260809 19:21:24.294950 10060 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 15 FOLLOWER]: Advancing to term 16
I20260809 19:21:24.295311 9652 raft_consensus.cc:1203] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 16 FOLLOWER]: Deduplicated request from leader. Original: 12.3916->[12.3917-15.4013] Dedup: 12.3917->[14.3918-15.4013]
I20260809 19:21:24.295461 9652 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 16 FOLLOWER]: Rejecting Update request from peer d4c4b93755eb4aa48c04795ad08e09ad for earlier term 15. Current term is 16. Ops: [12.3917-15.4013]
I20260809 19:21:24.298058 6380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:24.298158 6380 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 16.
I20260809 19:21:24.575019 10067 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d4c4b93755eb4aa48c04795ad08e09ad)
I20260809 19:21:24.575174 10067 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:24.575456 10067 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:24.575718 6380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:24.575834 6380 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 16.
I20260809 19:21:24.575790 9353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:24.578526 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:24.578635 9915 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 in current term 16: Already voted for candidate 757ec8723e924451a89abb5cc62c1073 in this term.
I20260809 19:21:24.972180 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:24.972342 5918 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 in current term 16: Already voted for candidate 757ec8723e924451a89abb5cc62c1073 in this term.
I20260809 19:21:24.973238 5479 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 16 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 14 index: 4012. Preceding OpId from leader: term: 16 index: 4014. (index mismatch)
I20260809 19:21:24.973356 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:24.973416 5482 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 in current term 16: Already voted for candidate 757ec8723e924451a89abb5cc62c1073 in this term.
I20260809 19:21:24.974515 9014 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 15 FOLLOWER]: Advancing to term 16
I20260809 19:21:24.975526 9014 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 16 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 15 index: 4013. Preceding OpId from leader: term: 16 index: 4014. (index mismatch)
I20260809 19:21:24.975656 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 16 candidate_status { last_received { term: 15 index: 4013 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:24.975769 9652 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 in current term 16: Already voted for candidate 757ec8723e924451a89abb5cc62c1073 in this term.
I20260809 19:21:24.976682 9915 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 16 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 15 index: 4013. Preceding OpId from leader: term: 16 index: 4014. (index mismatch)
I20260809 19:21:24.976608 10070 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4014, Last known committed idx: 4012, Time since last communication: 0.000s
I20260809 19:21:24.976861 10050 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4014, Last known committed idx: 4013, Time since last communication: 0.000s
I20260809 19:21:24.977201 9652 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 16 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 13 index: 3918. Preceding OpId from leader: term: 16 index: 4014. (index mismatch)
I20260809 19:21:24.977242 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 16 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 14 index: 4012. Preceding OpId from leader: term: 16 index: 4014. (index mismatch)
I20260809 19:21:24.977449 8935 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073
I20260809 19:21:24.977481 10050 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4014, Last known committed idx: 4013, Time since last communication: 0.000s
I20260809 19:21:24.977591 10050 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4014, Last known committed idx: 4012, Time since last communication: 0.000s
I20260809 19:21:24.977957 10067 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260809 19:21:24.978078 10070 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4014, Last known committed idx: 3916, Time since last communication: 0.000s
I20260809 19:21:24.978147 10070 consensus_queue.cc:1225] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Peer 61427f2036ac4f96a9655aef910a7d7a log is divergent from this leader: its last log entry 13.3918 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3916
I20260809 19:21:24.979182 6380 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 16 FOLLOWER]: Refusing update from remote peer 757ec8723e924451a89abb5cc62c1073: Log matching property violated. Preceding OpId in replica: term: 15 index: 4013. Preceding OpId from leader: term: 16 index: 4014. (index mismatch)
I20260809 19:21:24.979373 10070 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4014, Last known committed idx: 4013, Time since last communication: 0.000s
I20260809 19:21:24.980085 9652 raft_consensus.cc:1203] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 16 FOLLOWER]: Deduplicated request from leader. Original: 12.3916->[12.3917-16.4014] Dedup: 12.3917->[14.3918-16.4014]
I20260809 19:21:24.980180 9652 pending_rounds.cc:76] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Aborting all ops after (but not including) 3917
I20260809 19:21:24.980255 9652 pending_rounds.cc:97] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a: Aborting uncommitted NO_OP operation due to leader change: 13.3918
I20260809 19:21:24.980293 9652 raft_consensus.cc:2866] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 19:21:24.987988 10104 mvcc.cc:205] Tried to move back new op lower bound from 7316698234302062592 to 7316698233358901248. Current Snapshot: MvccSnapshot[applied={T|T < 7316698233942728704 or (T in {7316698233949769728,7316698233956020224,7316698233963130880,7316698233968730112,7316698233974599680,7316698233981689856,7316698234002186240,7316698233995214848,7316698234008092672,7316698234014400512,7316698234022039552,7316698233988329472,7316698234030657536,7316698234044518400,7316698234057768960,7316698234051153920,7316698234065846272,7316698234072494080,7316698234038296576,7316698234085240832,7316698234078834688,7316698234092593152,7316698234098651136,7316698234113818624,7316698234119663616,7316698234126127104,7316698234140184576,7316698234147717120,7316698234154188800,7316698234162704384,7316698234169434112,7316698234176737280,7316698234182782976,7316698234106454016,7316698234189119488,7316698234133401600,7316698234195488768,7316698234202783744,7316698234210799616})}]
I20260809 19:21:25.651036 4039 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKf1XAf/build/release/bin/kudu with pid 9212
W20260809 19:21:25.666611 6271 connection.cc:441] server connection from 127.3.241.197:35629 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.197:35629 (request call id 6) could send its response
W20260809 19:21:25.668008 5379 connection.cc:441] server connection from 127.3.241.197:49281 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.197:49281 (request call id 6) could send its response
W20260809 19:21:25.668457 8935 connection.cc:441] server connection from 127.3.241.197:58969 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.197:58969 (request call id 6) could send its response
W20260809 19:21:25.668905 9818 connection.cc:441] server connection from 127.3.241.197:49663 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.197:49663 (request call id 5) could send its response
W20260809 19:21:25.669353 9553 connection.cc:441] server connection from 127.3.241.197:52759 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.197:52759 (request call id 6) could send its response
W20260809 19:21:25.669466 5803 connection.cc:441] server connection from 127.3.241.197:55323 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.3.241.197:55323 (request call id 6) could send its response
I20260809 19:21:25.670100 4039 test_util.cc:268] Using random seed: 617459917
I20260809 19:21:25.774147 4039 test_util.cc:268] Using random seed: 617563961
I20260809 19:21:25.978276 4039 test_util.cc:268] Using random seed: 617768090
I20260809 19:21:26.282224 4039 test_util.cc:268] Using random seed: 618072030
I20260809 19:21:26.686383 4039 test_util.cc:268] Using random seed: 618476188
I20260809 19:21:27.166980 10131 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:27.167122 10131 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:27.167582 10131 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545)
I20260809 19:21:27.167850 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:27.167884 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:27.167970 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 16.
I20260809 19:21:27.168105 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:27.168213 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
W20260809 19:21:27.168196 6271 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260809 19:21:27.168301 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 16.
I20260809 19:21:27.168447 10133 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:27.168424 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:27.168519 10133 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:27.168814 10133 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:27.169070 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:27.169124 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:27.169220 9014 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 16.
I20260809 19:21:27.169265 10135 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:27.169238 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:27.169322 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 16.
I20260809 19:21:27.169296 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:27.169385 6379 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 16.
I20260809 19:21:27.169358 10135 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
W20260809 19:21:27.169564 6271 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:27.169618 6271 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:27.169802 5803 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters: 61427f2036ac4f96a9655aef910a7d7a
I20260809 19:21:27.169845 10141 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:27.169869 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:27.169916 10131 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260809 19:21:27.169852 8938 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260809 19:21:27.169963 9915 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 16.
I20260809 19:21:27.169950 10141 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
W20260809 19:21:27.170118 5805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260809 19:21:27.170281 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:27.170364 9652 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 16.
I20260809 19:21:27.170342 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:27.170419 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 16.
I20260809 19:21:27.170562 8934 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468; no voters:
I20260809 19:21:27.170614 10133 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260809 19:21:27.170428 10142 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:27.170665 10133 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:27.170693 10133 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 16 FOLLOWER]: Advancing to term 17
I20260809 19:21:27.170701 10142 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:27.170938 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:27.171077 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:27.171145 9652 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 16.
I20260809 19:21:27.171219 10142 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:27.171204 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:27.171270 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 16.
I20260809 19:21:27.171289 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:27.171360 6379 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 16.
I20260809 19:21:27.171473 10135 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:27.171538 9815 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad; no voters:
I20260809 19:21:27.171586 10135 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260809 19:21:27.171663 10135 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:27.171686 10135 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 16 FOLLOWER]: Advancing to term 17
I20260809 19:21:27.171646 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:27.171728 6379 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 16.
I20260809 19:21:27.171850 5922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:27.172017 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:27.172303 10141 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
W20260809 19:21:27.172713 9553 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:27.173036 10142 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260809 19:21:27.173205 10142 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 757ec8723e924451a89abb5cc62c1073)
I20260809 19:21:27.173257 10142 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 16 FOLLOWER]: Advancing to term 17
I20260809 19:21:27.173650 10135 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:27.173789 9014 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23183da38752444a9ff5525be50cd404 in current term 17: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:27.174081 10142 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:27.174357 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:27.174450 9915 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 17: Already voted for candidate 23183da38752444a9ff5525be50cd404 in this term.
I20260809 19:21:27.174467 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:27.174525 9652 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 16 FOLLOWER]: Advancing to term 17
I20260809 19:21:27.174495 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:21:27.174584 9014 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 23183da38752444a9ff5525be50cd404 in current term 17: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:27.174795 10135 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 17 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:27.174799 5922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:27.175349 9652 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 17.
W20260809 19:21:27.175375 5805 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:27.175707 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:27.175781 6379 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 16 FOLLOWER]: Advancing to term 17
I20260809 19:21:27.175848 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:27.175930 9915 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 in current term 17: Already voted for candidate 23183da38752444a9ff5525be50cd404 in this term.
I20260809 19:21:27.175978 10133 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:27.176201 10133 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 17 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:27.176194 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:27.176270 5482 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 16 FOLLOWER]: Advancing to term 17
I20260809 19:21:27.176622 6379 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 17.
I20260809 19:21:27.176942 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:21:27.177019 9014 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 17: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:27.177050 5918 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 in current term 17: Already voted for candidate 6c075d51984240f6b5d45d0d25a0d595 in this term.
I20260809 19:21:27.177121 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:27.177145 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 17.
I20260809 19:21:27.177464 9654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:27.177503 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:27.177548 9654 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 17: Already voted for candidate 23183da38752444a9ff5525be50cd404 in this term.
I20260809 19:21:27.177579 9915 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 17: Already voted for candidate 23183da38752444a9ff5525be50cd404 in this term.
I20260809 19:21:27.177820 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:27.177909 9652 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate dd68273a4b314386ae0f5489e2f1d468 in current term 17: Already voted for candidate 23183da38752444a9ff5525be50cd404 in this term.
W20260809 19:21:27.177917 9553 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:27.177989 5803 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:27.178084 10133 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260809 19:21:27.178094 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:27.178167 6379 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 17: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:27.178318 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:27.178404 9915 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a in current term 17: Already voted for candidate 23183da38752444a9ff5525be50cd404 in this term.
W20260809 19:21:27.178547 9817 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
W20260809 19:21:27.178699 8938 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
W20260809 19:21:27.178728 5805 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:27.179131 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:27.179211 5482 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 23183da38752444a9ff5525be50cd404 in current term 17: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:27.179589 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:27.179679 5918 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 23183da38752444a9ff5525be50cd404 in current term 17: Already voted for candidate 6c075d51984240f6b5d45d0d25a0d595 in this term.
W20260809 19:21:27.179793 8938 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:27.179838 8938 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073
I20260809 19:21:27.180173 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:27.180258 6379 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 23183da38752444a9ff5525be50cd404 in current term 17: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:27.180477 9815 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a; no voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:27.180671 10135 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260809 19:21:27.180691 10142 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 17 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:27.180794 10142 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260809 19:21:27.181352 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:27.181552 6379 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a in current term 17: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:27.182034 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:27.182355 9014 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a in current term 17: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:27.182665 9551 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a; no voters: 23183da38752444a9ff5525be50cd404, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:27.183032 10141 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260809 19:21:27.183477 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:27.183629 5918 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a in current term 17: Already voted for candidate 6c075d51984240f6b5d45d0d25a0d595 in this term.
W20260809 19:21:27.184335 9817 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
W20260809 19:21:27.184531 9817 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:27.185983 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 17 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:27.186100 5482 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61427f2036ac4f96a9655aef910a7d7a in current term 17: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:27.190125 4039 test_util.cc:268] Using random seed: 618979942
I20260809 19:21:27.794028 4039 test_util.cc:268] Using random seed: 619583835
I20260809 19:21:28.498135 4039 test_util.cc:268] Using random seed: 620287945
I20260809 19:21:28.684414 10181 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:28.684530 10181 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:28.684787 10181 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:28.685046 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 18 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:28.685046 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 18 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:28.685055 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 18 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:28.685146 9915 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 17.
I20260809 19:21:28.685146 6379 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 17.
I20260809 19:21:28.685179 9652 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 17.
I20260809 19:21:28.685361 8934 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters:
I20260809 19:21:28.685362 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 18 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:28.685459 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 17.
I20260809 19:21:28.685427 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 18 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:28.685531 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 17.
I20260809 19:21:28.685557 10181 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260809 19:21:28.685596 10181 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:21:28.685618 10181 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 17 FOLLOWER]: Advancing to term 18
W20260809 19:21:28.685777 8938 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:28.686401 10181 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:28.686591 10181 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 18 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:28.686733 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 18 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:28.686762 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 18 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:28.686811 5918 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 17 FOLLOWER]: Advancing to term 18
I20260809 19:21:28.686792 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 18 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:28.686836 5482 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 17 FOLLOWER]: Advancing to term 18
I20260809 19:21:28.686862 6379 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 17 FOLLOWER]: Advancing to term 18
I20260809 19:21:28.686841 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 18 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:28.686908 9652 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 17 FOLLOWER]: Advancing to term 18
I20260809 19:21:28.687639 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 18.
I20260809 19:21:28.687644 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 18.
I20260809 19:21:28.687644 6379 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 18.
I20260809 19:21:28.687743 9652 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 18.
I20260809 19:21:28.687757 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 18 candidate_status { last_received { term: 16 index: 4014 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:28.687839 9915 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 17 FOLLOWER]: Advancing to term 18
I20260809 19:21:28.687863 8934 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468; no voters:
I20260809 19:21:28.688079 10181 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 18 FOLLOWER]: Leader election won for term 18
W20260809 19:21:28.688144 8938 leader_election.cc:334] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111)
I20260809 19:21:28.688213 10181 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 18 LEADER]: Becoming Leader. State: Replica: dd68273a4b314386ae0f5489e2f1d468, State: Running, Role: LEADER
I20260809 19:21:28.688423 10181 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4014, Committed index: 4014, Last appended: 16.4014, Last appended by leader: 4014, Current term: 18, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:28.688694 9915 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 18.
I20260809 19:21:28.689311 5037 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 reported cstate change: term changed from 16 to 18, leader changed from 757ec8723e924451a89abb5cc62c1073 (127.3.241.197) to dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193). New cstate: current_term: 18 leader_uuid: "dd68273a4b314386ae0f5489e2f1d468" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
I20260809 19:21:29.123322 9915 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 18 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 16 index: 4014. Preceding OpId from leader: term: 18 index: 4015. (index mismatch)
I20260809 19:21:29.123692 10181 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4015, Last known committed idx: 4014, Time since last communication: 0.000s
W20260809 19:21:29.135507 8938 consensus_peers.cc:487] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 -> Peer 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111): Couldn't send request to peer 757ec8723e924451a89abb5cc62c1073. Status: Network error: Client connection negotiation failed: client connection to 127.3.241.197:45111: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 19:21:29.203388 6379 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 18 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 16 index: 4014. Preceding OpId from leader: term: 18 index: 4015. (index mismatch)
I20260809 19:21:29.203684 10181 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4015, Last known committed idx: 4014, Time since last communication: 0.000s
I20260809 19:21:29.285594 9652 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 18 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 16 index: 4014. Preceding OpId from leader: term: 18 index: 4015. (index mismatch)
I20260809 19:21:29.285903 10181 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4015, Last known committed idx: 4014, Time since last communication: 0.000s
I20260809 19:21:29.287855 5482 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 18 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 16 index: 4014. Preceding OpId from leader: term: 18 index: 4015. (index mismatch)
I20260809 19:21:29.288180 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 18 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 16 index: 4014. Preceding OpId from leader: term: 18 index: 4015. (index mismatch)
I20260809 19:21:29.288358 10214 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4015, Last known committed idx: 4014, Time since last communication: 0.000s
I20260809 19:21:29.288615 10219 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4015, Last known committed idx: 4014, Time since last communication: 0.000s
I20260809 19:21:29.302170 4039 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKf1XAf/build/release/bin/kudu
/tmp/dist-test-taskKf1XAf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.241.197:45111
--local_ip_for_outbound_sockets=127.3.241.197
--tserver_master_addrs=127.3.241.254:44125
--webserver_port=38231
--webserver_interface=127.3.241.197
--builtin_ntp_servers=127.3.241.212:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260809 19:21:29.388710 10235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:29.388921 10235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:29.388942 10235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:29.390138 10235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:29.390193 10235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.241.197
I20260809 19:21:29.391541 10235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.241.212:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.241.197:45111
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.3.241.197
--webserver_port=38231
--tserver_master_addrs=127.3.241.254:44125
--never_fsync=true
--heap_profile_path=/tmp/kudu.10235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.241.197
--log_dir=/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:29.391791 10235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:29.392109 10235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:29.397554 10250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:29.397675 10248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:29.397562 10246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:29.397785 10235 server_base.cc:1034] running on GCE node
I20260809 19:21:29.398037 10235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:29.398319 10235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:29.399478 10235 hybrid_clock.cc:648] HybridClock initialized: now 1786303289399462 us; error 33 us; skew 500 ppm
I20260809 19:21:29.400864 10235 webserver.cc:460] Webserver started at http://127.3.241.197:38231/ using document root <none> and password file <none>
I20260809 19:21:29.401118 10235 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:29.401187 10235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:29.402566 10235 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:29.403267 10256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:29.403438 10235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:29.403543 10235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "757ec8723e924451a89abb5cc62c1073"
format_stamp: "Formatted at 2026-08-09 19:20:49 on dist-test-slave-gmjp"
I20260809 19:21:29.403887 10235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:29.424326 10235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:29.424685 10235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:29.424839 10235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:29.425088 10235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:29.425575 10266 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:21:29.426870 10235 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:21:29.426932 10235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260809 19:21:29.426991 10235 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:21:29.427860 10235 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:21:29.427909 10235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 19:21:29.427956 10266 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Bootstrap starting.
I20260809 19:21:29.435962 10235 rpc_server.cc:307] RPC server started. Bound to: 127.3.241.197:45111
I20260809 19:21:29.436035 10438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.241.197:45111 every 8 connection(s)
I20260809 19:21:29.436480 10235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKf1XAf/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786303246403676-4039-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20260809 19:21:29.439106 4039 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKf1XAf/build/release/bin/kudu as pid 10235
I20260809 19:21:29.443513 10440 heartbeater.cc:347] Connected to a master server at 127.3.241.254:44125
I20260809 19:21:29.443619 10440 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:29.443805 10440 heartbeater.cc:511] Master 127.3.241.254:44125 requested a full tablet report, sending...
I20260809 19:21:29.444410 5036 ts_manager.cc:194] Re-registered known tserver with Master: 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111)
I20260809 19:21:29.444852 5036 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.241.197:35327
I20260809 19:21:29.492317 10266 log.cc:824] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:29.751475 10266 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Bootstrap replayed 1/1 log segments. Stats: ops{read=4038 overwritten=24 applied=4014 ignored=0} inserts{seen=40000 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:21:29.751945 10266 tablet_bootstrap.cc:492] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Bootstrap complete.
I20260809 19:21:29.753185 10266 ts_tablet_manager.cc:1397] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Time spent bootstrapping tablet: real 0.325s user 0.226s sys 0.096s
I20260809 19:21:29.754684 10266 raft_consensus.cc:348] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 16 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:29.754978 10266 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 16 FOLLOWER]: Becoming Follower/Learner. State: Replica: 757ec8723e924451a89abb5cc62c1073, State: Initialized, Role: FOLLOWER
I20260809 19:21:29.755155 10266 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4014, Last appended: 16.4014, Last appended by leader: 4014, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:29.755467 10266 ts_tablet_manager.cc:1428] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260809 19:21:29.755585 10440 heartbeater.cc:503] Master 127.3.241.254:44125 was elected leader, sending a full tablet report...
I20260809 19:21:30.214452 4039 test_util.cc:268] Using random seed: 622004264
I20260809 19:21:30.518874 4039 test_util.cc:268] Using random seed: 622308686
I20260809 19:21:30.923404 4039 test_util.cc:268] Using random seed: 622713215
I20260809 19:21:31.217255 10459 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:21:31.217370 10459 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:31.217654 10459 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:31.217906 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:31.217972 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:31.217963 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:31.218164 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:31.218427 9552 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a; no voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad
I20260809 19:21:31.218554 10459 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260809 19:21:31.221166 10373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:31.221392 10373 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 16.
I20260809 19:21:31.298029 10467 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:21:31.298151 10467 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:31.298409 10467 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:31.298666 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:31.298666 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:31.298770 9652 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 18.
I20260809 19:21:31.298671 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:31.299033 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:31.301640 10373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:31.301740 10373 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 16.
I20260809 19:21:31.428821 4039 test_util.cc:268] Using random seed: 623218634
I20260809 19:21:31.606369 10469 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:21:31.606547 10469 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:31.606835 10469 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545)
I20260809 19:21:31.607167 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:31.607146 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:31.607170 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:31.607261 9652 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 18.
I20260809 19:21:31.607304 9915 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 18.
I20260809 19:21:31.607445 10471 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:21:31.607569 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:31.607581 10471 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:31.607810 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 18.
I20260809 19:21:31.608033 10471 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:31.608137 6270 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 7d4177500c7a4f41afce5190ff786506
I20260809 19:21:31.608139 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:31.608242 9915 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 18.
I20260809 19:21:31.608243 10469 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260809 19:21:31.608274 10469 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 18 FOLLOWER]: Starting leader election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:21:31.608242 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:31.608294 10469 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 18 FOLLOWER]: Advancing to term 19
I20260809 19:21:31.608320 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 18.
I20260809 19:21:31.608305 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:31.608896 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:31.608996 9652 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 18.
I20260809 19:21:31.609138 5803 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506; no voters:
I20260809 19:21:31.609185 10469 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:31.609305 10471 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260809 19:21:31.609347 6379 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 19: Already voted for candidate d4c4b93755eb4aa48c04795ad08e09ad in this term.
I20260809 19:21:31.609375 10471 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 18 FOLLOWER]: Starting leader election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:21:31.609402 10471 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 18 FOLLOWER]: Advancing to term 19
I20260809 19:21:31.609512 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:31.609583 9915 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 18 FOLLOWER]: Advancing to term 19
I20260809 19:21:31.610291 9915 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 19.
I20260809 19:21:31.610186 10471 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:31.610425 10469 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 19 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545)
I20260809 19:21:31.610462 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:31.610492 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:31.610539 5918 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d4c4b93755eb4aa48c04795ad08e09ad in current term 19: Already voted for candidate 6c075d51984240f6b5d45d0d25a0d595 in this term.
I20260809 19:21:31.610559 9652 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 18 FOLLOWER]: Advancing to term 19
I20260809 19:21:31.610743 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:31.610816 9915 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 19: Already voted for candidate d4c4b93755eb4aa48c04795ad08e09ad in this term.
I20260809 19:21:31.610805 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:31.610903 5482 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 18 FOLLOWER]: Advancing to term 19
I20260809 19:21:31.611361 9652 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 19.
I20260809 19:21:31.611495 9654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:31.611583 9654 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 19: Already voted for candidate d4c4b93755eb4aa48c04795ad08e09ad in this term.
I20260809 19:21:31.611636 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c075d51984240f6b5d45d0d25a0d595 in term 19.
I20260809 19:21:31.611878 10471 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 19 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:31.611917 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:31.612006 5482 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d4c4b93755eb4aa48c04795ad08e09ad in current term 19: Already voted for candidate 6c075d51984240f6b5d45d0d25a0d595 in this term.
I20260809 19:21:31.612146 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:31.612254 6379 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c075d51984240f6b5d45d0d25a0d595 in current term 19: Already voted for candidate d4c4b93755eb4aa48c04795ad08e09ad in this term.
I20260809 19:21:31.614557 10373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:31.614670 10373 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 16.
I20260809 19:21:31.614629 10371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073"
I20260809 19:21:31.614755 10370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:31.614826 10373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073"
I20260809 19:21:31.614741 10371 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 16 FOLLOWER]: Advancing to term 19
I20260809 19:21:31.616137 10371 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4c4b93755eb4aa48c04795ad08e09ad in term 19.
I20260809 19:21:31.616259 5805 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad
I20260809 19:21:31.616369 10471 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260809 19:21:31.616374 6271 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506
I20260809 19:21:31.616530 10469 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 19 FOLLOWER]: Leader election won for term 19
I20260809 19:21:31.616642 10469 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 19 LEADER]: Becoming Leader. State: Replica: d4c4b93755eb4aa48c04795ad08e09ad, State: Running, Role: LEADER
I20260809 19:21:31.616711 10469 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4015, Committed index: 4015, Last appended: 18.4015, Last appended by leader: 4015, Current term: 19, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:31.617405 5037 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad reported cstate change: term changed from 18 to 19, leader changed from dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193) to d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198). New cstate: current_term: 19 leader_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: HEALTHY } } }
W20260809 19:21:31.933964 8934 connection.cc:232] RPC call timeout handler was delayed by -1.2575599423144013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260809 19:21:31.934106 8934 connection.cc:232] RPC call timeout handler was delayed by -1.145487260306254s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260809 19:21:31.934286 8935 connection.cc:232] RPC call timeout handler was delayed by -1.2117812861688435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260809 19:21:31.934373 9654 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 19 FOLLOWER]: Rejecting Update request from peer dd68273a4b314386ae0f5489e2f1d468 for earlier term 18. Current term is 19. Ops: []
I20260809 19:21:31.934504 9915 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 19 FOLLOWER]: Rejecting Update request from peer dd68273a4b314386ae0f5489e2f1d468 for earlier term 18. Current term is 19. Ops: []
I20260809 19:21:31.934707 9014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:31.934845 10486 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: INVALID_TERM, Last received: 18.4015, Next index: 4016, Last known committed idx: 4015, Time since last communication: 0.000s
I20260809 19:21:31.934882 9017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:31.934861 9016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:21:31.935015 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "6c075d51984240f6b5d45d0d25a0d595" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:21:31.935223 9031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:31.935438 9815 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073; no voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506, d4c4b93755eb4aa48c04795ad08e09ad, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:31.935554 10371 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 19 FOLLOWER]: Rejecting Update request from peer dd68273a4b314386ae0f5489e2f1d468 for earlier term 18. Current term is 19. Ops: [18.4015-18.4015]
I20260809 19:21:31.935632 10487 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260809 19:21:31.934973 10486 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: INVALID_TERM, Last received: 18.4015, Next index: 4016, Last known committed idx: 4015, Time since last communication: 0.000s
I20260809 19:21:31.936126 9027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 19 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:31.937341 10457 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4015, Last known committed idx: 4014, Time since last communication: 0.000s
I20260809 19:21:31.937410 10458 raft_consensus.cc:3032] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 18 LEADER]: Stepping down as leader of term 18
I20260809 19:21:31.937459 10458 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: dd68273a4b314386ae0f5489e2f1d468, State: Running, Role: LEADER
I20260809 19:21:31.937522 10458 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4015, Committed index: 4015, Last appended: 18.4015, Last appended by leader: 4015, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:31.937687 10458 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 18 FOLLOWER]: Advancing to term 19
I20260809 19:21:31.993337 9019 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 19 FOLLOWER]: Refusing update from remote peer d4c4b93755eb4aa48c04795ad08e09ad: Log matching property violated. Preceding OpId in replica: term: 18 index: 4015. Preceding OpId from leader: term: 19 index: 4016. (index mismatch)
I20260809 19:21:31.993741 10469 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4016, Last known committed idx: 4015, Time since last communication: 0.000s
I20260809 19:21:32.032347 9654 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 19 FOLLOWER]: Refusing update from remote peer d4c4b93755eb4aa48c04795ad08e09ad: Log matching property violated. Preceding OpId in replica: term: 18 index: 4015. Preceding OpId from leader: term: 19 index: 4016. (index mismatch)
I20260809 19:21:32.032698 10469 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4016, Last known committed idx: 4015, Time since last communication: 0.000s
I20260809 19:21:32.183074 10371 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 19 FOLLOWER]: Refusing update from remote peer d4c4b93755eb4aa48c04795ad08e09ad: Log matching property violated. Preceding OpId in replica: term: 16 index: 4014. Preceding OpId from leader: term: 19 index: 4016. (index mismatch)
I20260809 19:21:32.183416 10469 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4016, Last known committed idx: 4014, Time since last communication: 0.000s
I20260809 19:21:32.185485 9915 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 19 FOLLOWER]: Refusing update from remote peer d4c4b93755eb4aa48c04795ad08e09ad: Log matching property violated. Preceding OpId in replica: term: 18 index: 4015. Preceding OpId from leader: term: 19 index: 4016. (index mismatch)
I20260809 19:21:32.185827 10504 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4016, Last known committed idx: 4015, Time since last communication: 0.000s
I20260809 19:21:32.613677 5482 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 19 FOLLOWER]: Refusing update from remote peer d4c4b93755eb4aa48c04795ad08e09ad: Log matching property violated. Preceding OpId in replica: term: 18 index: 4015. Preceding OpId from leader: term: 19 index: 4016. (index mismatch)
I20260809 19:21:32.615180 5918 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 19 FOLLOWER]: Refusing update from remote peer d4c4b93755eb4aa48c04795ad08e09ad: Log matching property violated. Preceding OpId in replica: term: 18 index: 4015. Preceding OpId from leader: term: 19 index: 4016. (index mismatch)
I20260809 19:21:32.719391 4039 test_util.cc:268] Using random seed: 624509203
I20260809 19:21:33.024287 4039 test_util.cc:268] Using random seed: 624814099
I20260809 19:21:33.400003 10512 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:33.400117 10512 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:33.400432 10512 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:33.400695 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 20 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:33.400810 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d4177500c7a4f41afce5190ff786506 in term 19.
I20260809 19:21:33.400784 9654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 20 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:33.400907 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 20 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:33.400996 9019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 20 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:33.403714 10371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 20 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:33.403975 5379 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 7d4177500c7a4f41afce5190ff786506; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 757ec8723e924451a89abb5cc62c1073, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:33.404217 10512 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260809 19:21:33.428480 4039 test_util.cc:268] Using random seed: 625218293
I20260809 19:21:33.685587 10519 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d4c4b93755eb4aa48c04795ad08e09ad)
I20260809 19:21:33.685698 10519 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:33.686092 10520 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d4c4b93755eb4aa48c04795ad08e09ad)
I20260809 19:21:33.686167 10519 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:33.686231 10520 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:33.686805 9654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:33.687294 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:33.687297 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:33.687402 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 19.
I20260809 19:21:33.687615 10371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:33.687696 10371 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 19.
I20260809 19:21:33.688067 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:33.688215 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 19.
I20260809 19:21:33.688455 8935 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a
I20260809 19:21:33.688725 10520 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:33.688841 10520 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260809 19:21:33.688905 10520 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d4c4b93755eb4aa48c04795ad08e09ad)
I20260809 19:21:33.688964 10520 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 19 FOLLOWER]: Advancing to term 20
I20260809 19:21:33.689937 10520 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:33.690292 10371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073"
I20260809 19:21:33.690367 10371 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 19 FOLLOWER]: Advancing to term 20
I20260809 19:21:33.691120 10371 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 20.
I20260809 19:21:33.691517 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:33.691919 9654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a"
I20260809 19:21:33.692369 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:33.692477 5482 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 19 FOLLOWER]: Advancing to term 20
I20260809 19:21:33.693210 5482 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 20.
I20260809 19:21:33.693500 9019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:33.693614 9019 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 757ec8723e924451a89abb5cc62c1073 in current term 20: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:33.693768 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:33.693789 10520 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 20 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:33.693950 5918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:33.693984 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:33.694061 5482 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 757ec8723e924451a89abb5cc62c1073 in current term 20: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:33.694044 5918 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 19.
I20260809 19:21:33.694137 5922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:33.694195 5922 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 19 FOLLOWER]: Advancing to term 20
I20260809 19:21:33.694221 9654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:33.694484 10257 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:33.694638 10519 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260809 19:21:33.694988 5922 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd68273a4b314386ae0f5489e2f1d468 in term 20.
I20260809 19:21:33.695151 8935 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468; no voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a
I20260809 19:21:33.695261 10520 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 20 FOLLOWER]: Leader election won for term 20
I20260809 19:21:33.695324 10520 raft_consensus.cc:686] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 20 LEADER]: Becoming Leader. State: Replica: dd68273a4b314386ae0f5489e2f1d468, State: Running, Role: LEADER
I20260809 19:21:33.695382 10520 consensus_queue.cc:237] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4016, Committed index: 4016, Last appended: 19.4016, Last appended by leader: 4016, Current term: 20, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:33.696099 5037 catalog_manager.cc:5495] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 reported cstate change: term changed from 19 to 20, leader changed from d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198) to dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193). New cstate: current_term: 20 leader_uuid: "dd68273a4b314386ae0f5489e2f1d468" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } health_report { overall_health: UNKNOWN } } }
I20260809 19:21:33.833472 6380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:33.833631 6376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "7d4177500c7a4f41afce5190ff786506" candidate_term: 20 candidate_status { last_received { term: 18 index: 4015 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:33.833787 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "dd68273a4b314386ae0f5489e2f1d468" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:33.834128 10549 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4016, Last known committed idx: 4015, Time since last communication: 0.000s
I20260809 19:21:33.834254 10550 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4016, Last known committed idx: 4015, Time since last communication: 0.000s
I20260809 19:21:33.834599 5922 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 20 FOLLOWER]: Rejecting Update request from peer d4c4b93755eb4aa48c04795ad08e09ad for earlier term 19. Current term is 20. Ops: [19.4016-19.4016]
I20260809 19:21:33.835026 10371 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 20 FOLLOWER]: Rejecting Update request from peer d4c4b93755eb4aa48c04795ad08e09ad for earlier term 19. Current term is 20. Ops: []
I20260809 19:21:33.835168 10469 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: INVALID_TERM, Last received: 18.4015, Next index: 4016, Last known committed idx: 4015, Time since last communication: 0.000s
I20260809 19:21:33.835256 10550 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: INVALID_TERM, Last received: 19.4016, Next index: 4017, Last known committed idx: 4016, Time since last communication: 0.000s
I20260809 19:21:33.835311 10469 raft_consensus.cc:3032] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 19 LEADER]: Stepping down as leader of term 19
I20260809 19:21:33.835351 10469 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d4c4b93755eb4aa48c04795ad08e09ad, State: Running, Role: LEADER
I20260809 19:21:33.835421 10469 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4016, Committed index: 4016, Last appended: 19.4016, Last appended by leader: 4016, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:33.835533 10469 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 19 FOLLOWER]: Advancing to term 20
I20260809 19:21:33.836969 9019 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 20 LEADER]: Rejecting Update request from peer d4c4b93755eb4aa48c04795ad08e09ad for earlier term 19. Current term is 20. Ops: []
I20260809 19:21:33.837239 5482 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 20 FOLLOWER]: Rejecting Update request from peer d4c4b93755eb4aa48c04795ad08e09ad for earlier term 19. Current term is 20. Ops: [19.4016-19.4016]
I20260809 19:21:33.838675 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "757ec8723e924451a89abb5cc62c1073" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:33.838745 6379 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 757ec8723e924451a89abb5cc62c1073 in term 20.
I20260809 19:21:34.118431 10552 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d4c4b93755eb4aa48c04795ad08e09ad)
I20260809 19:21:34.118563 10552 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:34.118824 10552 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 61427f2036ac4f96a9655aef910a7d7a (127.3.241.195:36997), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:34.119052 9019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:34.119042 10371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:34.119184 10371 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23183da38752444a9ff5525be50cd404 in current term 20: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:34.119112 6379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:34.119235 6379 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 20.
I20260809 19:21:34.135711 6379 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P d4c4b93755eb4aa48c04795ad08e09ad [term 20 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 19 index: 4016. Preceding OpId from leader: term: 20 index: 4017. (index mismatch)
I20260809 19:21:34.136030 10520 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4017, Last known committed idx: 4016, Time since last communication: 0.000s
I20260809 19:21:34.514457 5922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:34.514601 5922 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23183da38752444a9ff5525be50cd404 in current term 20: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:34.516008 9652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "61427f2036ac4f96a9655aef910a7d7a" is_pre_election: true
I20260809 19:21:34.516109 9652 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23183da38752444a9ff5525be50cd404 in term 19.
I20260809 19:21:34.516093 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "23183da38752444a9ff5525be50cd404" candidate_term: 20 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:34.516268 5482 raft_consensus.cc:2379] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 23183da38752444a9ff5525be50cd404 in current term 20: Already voted for candidate dd68273a4b314386ae0f5489e2f1d468 in this term.
I20260809 19:21:34.516464 9654 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 19 FOLLOWER]: Advancing to term 20
I20260809 19:21:34.516817 5479 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 7d4177500c7a4f41afce5190ff786506 [term 20 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 18 index: 4015. Preceding OpId from leader: term: 20 index: 4017. (index mismatch)
I20260809 19:21:34.517421 9654 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 20 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 19 index: 4016. Preceding OpId from leader: term: 20 index: 4017. (index mismatch)
I20260809 19:21:34.518478 9815 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 23183da38752444a9ff5525be50cd404, d4c4b93755eb4aa48c04795ad08e09ad; no voters: 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506, dd68273a4b314386ae0f5489e2f1d468
I20260809 19:21:34.518815 10520 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4017, Last known committed idx: 4016, Time since last communication: 0.000s
I20260809 19:21:34.518997 10520 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4017, Last known committed idx: 4015, Time since last communication: 0.000s
I20260809 19:21:34.523170 10552 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 19 FOLLOWER]: Advancing to term 20
I20260809 19:21:34.523589 10556 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:34.523650 10556 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:34.524438 10552 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260809 19:21:34.524679 10371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 21 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073" is_pre_election: true
I20260809 19:21:34.524785 10371 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 20.
I20260809 19:21:34.524889 9915 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 20 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 19 index: 4016. Preceding OpId from leader: term: 20 index: 4017. (index mismatch)
I20260809 19:21:34.525192 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 21 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404" is_pre_election: true
I20260809 19:21:34.525275 9915 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 23183da38752444a9ff5525be50cd404 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 20.
I20260809 19:21:34.525242 5482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 21 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506" is_pre_election: true
I20260809 19:21:34.525516 5922 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 20 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 18 index: 4015. Preceding OpId from leader: term: 20 index: 4017. (index mismatch)
I20260809 19:21:34.525520 10556 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:34.525789 9019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 21 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468" is_pre_election: true
I20260809 19:21:34.525911 10560 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4017, Last known committed idx: 4016, Time since last communication: 0.000s
I20260809 19:21:34.526021 5922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 21 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595" is_pre_election: true
I20260809 19:21:34.526098 5922 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P 6c075d51984240f6b5d45d0d25a0d595 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 20.
I20260809 19:21:34.526963 10371 raft_consensus.cc:1261] T 7c092d80045840deac3151a94e01de0b P 757ec8723e924451a89abb5cc62c1073 [term 20 FOLLOWER]: Refusing update from remote peer dd68273a4b314386ae0f5489e2f1d468: Log matching property violated. Preceding OpId in replica: term: 19 index: 4016. Preceding OpId from leader: term: 20 index: 4017. (index mismatch)
I20260809 19:21:34.527012 9552 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 23183da38752444a9ff5525be50cd404, 61427f2036ac4f96a9655aef910a7d7a, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073; no voters: 7d4177500c7a4f41afce5190ff786506
I20260809 19:21:34.527163 6372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 21 candidate_status { last_received { term: 19 index: 4016 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" is_pre_election: true
I20260809 19:21:34.527186 10556 raft_consensus.cc:2781] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260809 19:21:34.527765 10560 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4017, Last known committed idx: 4016, Time since last communication: 0.000s
I20260809 19:21:34.527271 10556 raft_consensus.cc:482] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 20 FOLLOWER]: Starting leader election (detected failure of leader dd68273a4b314386ae0f5489e2f1d468)
I20260809 19:21:34.528653 10556 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 20 FOLLOWER]: Advancing to term 21
I20260809 19:21:34.527935 10560 consensus_queue.cc:1035] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4017, Last known committed idx: 4015, Time since last communication: 0.000s
I20260809 19:21:34.530666 10556 raft_consensus.cc:504] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:34.530814 9654 raft_consensus.cc:1226] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 21 FOLLOWER]: Rejecting Update request from peer dd68273a4b314386ae0f5489e2f1d468 for earlier term 20. Current term is 21. Ops: []
I20260809 19:21:34.531445 10580 consensus_queue.cc:1046] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 }, Status: INVALID_TERM, Last received: 20.4017, Next index: 4018, Last known committed idx: 4016, Time since last communication: 0.000s
I20260809 19:21:34.531491 10371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 21 candidate_status { last_received { term: 20 index: 4017 } } ignore_live_leader: false dest_uuid: "757ec8723e924451a89abb5cc62c1073"
I20260809 19:21:34.531594 10580 raft_consensus.cc:3032] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 20 LEADER]: Stepping down as leader of term 20
I20260809 19:21:34.531553 9915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 21 candidate_status { last_received { term: 20 index: 4017 } } ignore_live_leader: false dest_uuid: "23183da38752444a9ff5525be50cd404"
I20260809 19:21:34.531630 10580 raft_consensus.cc:729] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: dd68273a4b314386ae0f5489e2f1d468, State: Running, Role: LEADER
I20260809 19:21:34.531677 10580 consensus_queue.cc:260] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4017, Committed index: 4017, Last appended: 20.4017, Last appended by leader: 4017, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "757ec8723e924451a89abb5cc62c1073" member_type: VOTER last_known_addr { host: "127.3.241.197" port: 45111 } } peers { permanent_uuid: "23183da38752444a9ff5525be50cd404" member_type: VOTER last_known_addr { host: "127.3.241.199" port: 41771 } } peers { permanent_uuid: "dd68273a4b314386ae0f5489e2f1d468" member_type: VOTER last_known_addr { host: "127.3.241.193" port: 46757 } } peers { permanent_uuid: "61427f2036ac4f96a9655aef910a7d7a" member_type: VOTER last_known_addr { host: "127.3.241.195" port: 36997 } } peers { permanent_uuid: "7d4177500c7a4f41afce5190ff786506" member_type: VOTER last_known_addr { host: "127.3.241.194" port: 41469 } } peers { permanent_uuid: "6c075d51984240f6b5d45d0d25a0d595" member_type: VOTER last_known_addr { host: "127.3.241.196" port: 35545 } } peers { permanent_uuid: "d4c4b93755eb4aa48c04795ad08e09ad" member_type: VOTER last_known_addr { host: "127.3.241.198" port: 39597 } }
I20260809 19:21:34.531795 10580 raft_consensus.cc:3037] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 20 FOLLOWER]: Advancing to term 21
I20260809 19:21:34.531996 5479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 21 candidate_status { last_received { term: 20 index: 4017 } } ignore_live_leader: false dest_uuid: "7d4177500c7a4f41afce5190ff786506"
I20260809 19:21:34.532637 9019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 21 candidate_status { last_received { term: 20 index: 4017 } } ignore_live_leader: false dest_uuid: "dd68273a4b314386ae0f5489e2f1d468"
I20260809 19:21:34.532660 5922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 21 candidate_status { last_received { term: 20 index: 4017 } } ignore_live_leader: false dest_uuid: "6c075d51984240f6b5d45d0d25a0d595"
I20260809 19:21:34.532995 6372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c092d80045840deac3151a94e01de0b" candidate_uuid: "61427f2036ac4f96a9655aef910a7d7a" candidate_term: 21 candidate_status { last_received { term: 20 index: 4017 } } ignore_live_leader: false dest_uuid: "d4c4b93755eb4aa48c04795ad08e09ad"
I20260809 19:21:34.533041 9552 leader_election.cc:302] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 61427f2036ac4f96a9655aef910a7d7a; no voters: 23183da38752444a9ff5525be50cd404, 6c075d51984240f6b5d45d0d25a0d595, 757ec8723e924451a89abb5cc62c1073, 7d4177500c7a4f41afce5190ff786506
I20260809 19:21:34.533121 10564 raft_consensus.cc:2726] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260809 19:21:34.533416 10556 leader_election.cc:288] T 7c092d80045840deac3151a94e01de0b P 61427f2036ac4f96a9655aef910a7d7a [CANDIDATE]: Term 21 election: Requested vote from peers 757ec8723e924451a89abb5cc62c1073 (127.3.241.197:45111), 23183da38752444a9ff5525be50cd404 (127.3.241.199:41771), dd68273a4b314386ae0f5489e2f1d468 (127.3.241.193:46757), 7d4177500c7a4f41afce5190ff786506 (127.3.241.194:41469), 6c075d51984240f6b5d45d0d25a0d595 (127.3.241.196:35545), d4c4b93755eb4aa48c04795ad08e09ad (127.3.241.198:39597)
I20260809 19:21:34.533573 9019 raft_consensus.cc:2454] T 7c092d80045840deac3151a94e01de0b P dd68273a4b314386ae0f5489e2f1d468 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61427f2036ac4f96a9655aef910a7d7a in term 21.
F20260809 19:21:34.565196 4039 raft_consensus-itest.cc:398] Check failed: _s.ok() Bad status: Not found: leader replica not found
*** Check failure stack trace: ***
*** Aborted at 1786303294 (unix time) try "date -d @1786303294" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800000fc7) received by PID 4039 (TID 0x7f0dba8df440) from PID 4039; stack trace: ***
@ 0x7f0dbdab5a46 google::(anonymous namespace)::FailureSignalHandler() at ??:0
@ 0x7f0dbf87f980 (unknown) at ??:0
@ 0x7f0dbb5cde87 gsignal at ??:0
@ 0x7f0dbb5cf7f1 abort at ??:0
@ 0x7f0dbdaa79ad google::LogMessage::Fail() at ??:0
@ 0x7f0dbdaaa88a google::LogMessage::SendToLog() at ??:0
@ 0x7f0dbdaa74de google::LogMessage::Flush() at ??:0
@ 0x7f0dbdaa8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x43b95e kudu::tserver::RaftConsensusITest::StopOrKillLeaderAndElectNewOne() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/raft_consensus-itest.cc:435 (discriminator 21)
@ 0x446ccf kudu::tserver::RaftConsensusITest_MultiThreadedInsertWithFailovers_Test::TestBody() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/raft_consensus-itest.cc:1060
@ 0x7f0dbeec1bfe testing::internal::HandleExceptionsInMethodIfSupported<>() at ??:0
@ 0x7f0dbeeb65db testing::Test::Run() at ??:0
@ 0x7f0dbeeb6795 testing::TestInfo::Run() at ??:0
@ 0x7f0dbeeb6cdf testing::TestSuite::Run() at ??:0
@ 0x7f0dbeeb729f testing::internal::UnitTestImpl::RunAllTests() at ??:0
@ 0x7f0dbeec213e testing::internal::HandleExceptionsInMethodIfSupported<>() at ??:0
@ 0x7f0dbeeb683d testing::UnitTest::Run() at ??:0
@ 0x438cba main at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/test_main.cc:110
@ 0x7f0dbb5b0c87 __libc_start_main at ??:0
@ 0x438ef2 (unknown) at ??:?